[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:55:45 dist-test-slave-mmhs krb5kdc[8457](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:55:45 dist-test-slave-mmhs krb5kdc[8457](info): set up 2 sockets
Sep 24 00:55:45 dist-test-slave-mmhs krb5kdc[8457](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260924 00:55:46.367130 8450 mini_kdc.cc:121] Time spent starting KDC: real 3.181s user 0.000s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260924 00:55:46.527130 8450 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.160s user 0.000s sys 0.004s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260924 00:55:46.760783 8450 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.233s user 0.000s sys 0.004s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260924 00:55:46.996415 8450 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.235s user 0.000s sys 0.004s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/krb5kdc/test-user.keytab.
W20260924 00:55:47.278126 8450 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.281s user 0.000s sys 0.004s
Sep 24 00:55:47 dist-test-slave-mmhs krb5kdc[8457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211347, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:55:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:55:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.64.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
I20260924 00:55:47.358763 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:37613
--webserver_interface=127.8.64.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:41613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:37613
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:55:47.685822 8470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:55:47.686178 8470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:55:47.686406 8470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:55:47.693583 8470 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:55:47.693723 8470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:55:47.693827 8470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:55:47.693933 8470 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:55:47.694021 8470 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:55:47.704047 8470 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:41613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:37613
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:37613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:55:47.705427 8470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:55:47.706826 8470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:55:47.719069 8475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:55:47.719096 8476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:55:47.719123 8478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:55:47.719515 8470 server_base.cc:1034] running on GCE node
I20260924 00:55:47.720548 8470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:55:47.721946 8470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:55:47.723197 8470 hybrid_clock.cc:648] HybridClock initialized: now 1790211347723152 us; error 34 us; skew 500 ppm
I20260924 00:55:47.723574 8470 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:55:47 dist-test-slave-mmhs krb5kdc[8457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211347, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:55:47.726742 8470 init.cc:377] Logged in from keytab as kudu/127.8.64.190@KRBTEST.COM (short username kudu)
I20260924 00:55:47.728232 8470 webserver.cc:460] Webserver started at http://127.8.64.190:34461/ using document root <none> and password file <none>
I20260924 00:55:47.728951 8470 fs_manager.cc:359] Metadata directory not provided
I20260924 00:55:47.729101 8470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:55:47.729410 8470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:55:47.732453 8470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "00d86d5f63b946ac8b6e1a1a5b414932"
format_stamp: "Formatted at 2026-09-24 00:55:47 on dist-test-slave-mmhs"
server_key: "3579e0c199ea6cee8bd76d4c7537310a"
server_key_iv: "7e0b8933c12117584c00b0753bd8fd00"
server_key_version: "encryptionkey@0"
I20260924 00:55:47.733196 8470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "00d86d5f63b946ac8b6e1a1a5b414932"
format_stamp: "Formatted at 2026-09-24 00:55:47 on dist-test-slave-mmhs"
server_key: "3579e0c199ea6cee8bd76d4c7537310a"
server_key_iv: "7e0b8933c12117584c00b0753bd8fd00"
server_key_version: "encryptionkey@0"
I20260924 00:55:47.738555 8470 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260924 00:55:47.741986 8485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:55:47.743348 8470 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 00:55:47.743554 8470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "00d86d5f63b946ac8b6e1a1a5b414932"
format_stamp: "Formatted at 2026-09-24 00:55:47 on dist-test-slave-mmhs"
server_key: "3579e0c199ea6cee8bd76d4c7537310a"
server_key_iv: "7e0b8933c12117584c00b0753bd8fd00"
server_key_version: "encryptionkey@0"
I20260924 00:55:47.743754 8470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:55:47.768113 8470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:55:47.772210 8470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:55:47.772516 8470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:55:47.790707 8470 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.190:37613
I20260924 00:55:47.790750 8536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.190:37613 every 8 connection(s)
I20260924 00:55:47.792059 8470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 00:55:47.795385 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 8470
I20260924 00:55:47.795732 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 00:55:47.796336 8537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:55:47.798277 8450 external_mini_cluster.cc:1428] Setting key 1f53caebb3c046c4a1fd47665f1d1b20
I20260924 00:55:47.810844 8537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00d86d5f63b946ac8b6e1a1a5b414932: Bootstrap starting.
I20260924 00:55:47.815003 8537 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00d86d5f63b946ac8b6e1a1a5b414932: Neither blocks nor log segments found. Creating new log.
I20260924 00:55:47.816155 8537 log.cc:824] T 00000000000000000000000000000000 P 00d86d5f63b946ac8b6e1a1a5b414932: Log is configured to *not* fsync() on all Append() calls
I20260924 00:55:47.820086 8537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00d86d5f63b946ac8b6e1a1a5b414932: No bootstrap required, opened a new log
I20260924 00:55:47.830583 8537 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00d86d5f63b946ac8b6e1a1a5b414932 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00d86d5f63b946ac8b6e1a1a5b414932" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 37613 } }
I20260924 00:55:47.831066 8537 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00d86d5f63b946ac8b6e1a1a5b414932 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:55:47.831174 8537 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00d86d5f63b946ac8b6e1a1a5b414932 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00d86d5f63b946ac8b6e1a1a5b414932, State: Initialized, Role: FOLLOWER
I20260924 00:55:47.831892 8537 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00d86d5f63b946ac8b6e1a1a5b414932 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00d86d5f63b946ac8b6e1a1a5b414932" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 37613 } }
I20260924 00:55:47.832188 8537 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00d86d5f63b946ac8b6e1a1a5b414932 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:55:47.832331 8537 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00d86d5f63b946ac8b6e1a1a5b414932 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:55:47.832528 8537 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00d86d5f63b946ac8b6e1a1a5b414932 [term 0 FOLLOWER]: Advancing to term 1
Sep 24 00:55:47 dist-test-slave-mmhs krb5kdc[8457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211347, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:55:47.835060 8537 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00d86d5f63b946ac8b6e1a1a5b414932 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00d86d5f63b946ac8b6e1a1a5b414932" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 37613 } }
I20260924 00:55:47.835711 8537 leader_election.cc:302] T 00000000000000000000000000000000 P 00d86d5f63b946ac8b6e1a1a5b414932 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00d86d5f63b946ac8b6e1a1a5b414932; no voters:
I20260924 00:55:47.836434 8537 leader_election.cc:288] T 00000000000000000000000000000000 P 00d86d5f63b946ac8b6e1a1a5b414932 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:55:47.836565 8542 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00d86d5f63b946ac8b6e1a1a5b414932 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:55:47.838182 8542 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00d86d5f63b946ac8b6e1a1a5b414932 [term 1 LEADER]: Becoming Leader. State: Replica: 00d86d5f63b946ac8b6e1a1a5b414932, State: Running, Role: LEADER
I20260924 00:55:47.838806 8542 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00d86d5f63b946ac8b6e1a1a5b414932 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00d86d5f63b946ac8b6e1a1a5b414932" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 37613 } }
I20260924 00:55:47.838995 8537 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00d86d5f63b946ac8b6e1a1a5b414932 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:55:47.841388 8544 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00d86d5f63b946ac8b6e1a1a5b414932 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00d86d5f63b946ac8b6e1a1a5b414932. Latest consensus state: current_term: 1 leader_uuid: "00d86d5f63b946ac8b6e1a1a5b414932" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00d86d5f63b946ac8b6e1a1a5b414932" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 37613 } } }
I20260924 00:55:47.841760 8544 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00d86d5f63b946ac8b6e1a1a5b414932 [sys.catalog]: This master's current role is: LEADER
I20260924 00:55:47.842414 8543 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00d86d5f63b946ac8b6e1a1a5b414932 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00d86d5f63b946ac8b6e1a1a5b414932" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00d86d5f63b946ac8b6e1a1a5b414932" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 37613 } } }
I20260924 00:55:47.842764 8543 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00d86d5f63b946ac8b6e1a1a5b414932 [sys.catalog]: This master's current role is: LEADER
I20260924 00:55:47.845099 8547 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:55:47.847038 8539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:47.802900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43944 (local address 127.8.64.190:37613)
0924 00:55:47.803842 (+ 942us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:47.803860 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:47.805082 (+ 1222us) server_negotiation.cc:407] Connection header received
0924 00:55:47.817458 (+ 12376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:47.817491 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:47.817972 (+ 481us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:47.818377 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:55:47.822785 (+ 4408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:47.824320 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:47.826836 (+ 2516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:47.827527 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:47.827654 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:47.834562 (+ 6908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:55:47.834572 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:55:47.834591 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:55:47.834640 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:55:47.838954 (+ 4314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:47.840105 (+ 1151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:47.840113 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:47.840122 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:47.840526 (+ 404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:47.841276 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:47.841284 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:47.841290 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:47.842186 (+ 896us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:55:47.845198 (+ 3012us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:47.845444 (+ 246us) server_negotiation.cc:299] Negotiation successful
0924 00:55:47.845729 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":380,"threads_started":1}
I20260924 00:55:47.851235 8547 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:55:47.864096 8547 catalog_manager.cc:1347] Generated new cluster ID: 2ceae0e582f346da8b7636463aa6f536
I20260924 00:55:47.864207 8547 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:55:47.877609 8547 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:55:47.878944 8547 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:55:47.887244 8547 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00d86d5f63b946ac8b6e1a1a5b414932: Generated new TSK 0
I20260924 00:55:47.888111 8547 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.64.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab.
Entry for principal kudu/127.8.64.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab.
I20260924 00:55:47.934566 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.129:0
--local_ip_for_outbound_sockets=127.8.64.129
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:37613
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab
--principal=kudu/127.8.64.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:41613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:55:48.259974 8564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:55:48.260236 8564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:55:48.260330 8564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:55:48.267370 8564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:55:48.267474 8564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:55:48.267592 8564 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.129
I20260924 00:55:48.277666 8564 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:41613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab
--principal=kudu/127.8.64.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:37613
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.129
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:55:48.279006 8564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:55:48.280251 8564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:55:48.292984 8569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:55:48.293037 8572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:55:48.293073 8570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:55:48.293704 8564 server_base.cc:1034] running on GCE node
I20260924 00:55:48.294214 8564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:55:48.295146 8564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:55:48.296391 8564 hybrid_clock.cc:648] HybridClock initialized: now 1790211348296338 us; error 33 us; skew 500 ppm
I20260924 00:55:48.296761 8564 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:55:48 dist-test-slave-mmhs krb5kdc[8457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211348, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:55:48.300711 8564 init.cc:377] Logged in from keytab as kudu/127.8.64.129@KRBTEST.COM (short username kudu)
I20260924 00:55:48.306973 8564 webserver.cc:460] Webserver started at http://127.8.64.129:34443/ using document root <none> and password file <none>
I20260924 00:55:48.307672 8564 fs_manager.cc:359] Metadata directory not provided
I20260924 00:55:48.307816 8564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:55:48.308261 8564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:55:48.311480 8564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-0/data/instance:
uuid: "ce7ea4b474f24db0addebb82965fdd6c"
format_stamp: "Formatted at 2026-09-24 00:55:48 on dist-test-slave-mmhs"
server_key: "8be95365dfa330f0dcc8b96e464a2d16"
server_key_iv: "3ce6ba72f8e1d969b9a8b7d379c08bc2"
server_key_version: "encryptionkey@0"
I20260924 00:55:48.312294 8564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance:
uuid: "ce7ea4b474f24db0addebb82965fdd6c"
format_stamp: "Formatted at 2026-09-24 00:55:48 on dist-test-slave-mmhs"
server_key: "8be95365dfa330f0dcc8b96e464a2d16"
server_key_iv: "3ce6ba72f8e1d969b9a8b7d379c08bc2"
server_key_version: "encryptionkey@0"
I20260924 00:55:48.317687 8564 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260924 00:55:48.321535 8579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:55:48.323001 8564 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 00:55:48.323185 8564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-0/wal
uuid: "ce7ea4b474f24db0addebb82965fdd6c"
format_stamp: "Formatted at 2026-09-24 00:55:48 on dist-test-slave-mmhs"
server_key: "8be95365dfa330f0dcc8b96e464a2d16"
server_key_iv: "3ce6ba72f8e1d969b9a8b7d379c08bc2"
server_key_version: "encryptionkey@0"
I20260924 00:55:48.323391 8564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:55:48.344059 8564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:55:48.348949 8564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:55:48.349273 8564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:55:48.350235 8564 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:55:48.351579 8564 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:55:48.351655 8564 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:55:48.351817 8564 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:55:48.351877 8564 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:55:48.384706 8564 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.129:38999
I20260924 00:55:48.384792 8691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.129:38999 every 8 connection(s)
I20260924 00:55:48.385982 8564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
I20260924 00:55:48.391920 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 8564
I20260924 00:55:48.392172 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance
I20260924 00:55:48.392795 8450 external_mini_cluster.cc:1428] Setting key a1c3794ff5891adaf6e293446c60073c
Sep 24 00:55:48 dist-test-slave-mmhs krb5kdc[8457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211348, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:55:48.406116 8694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:48.388803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:52207 (local address 127.8.64.190:37613)
0924 00:55:48.389224 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:48.389228 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:48.390029 (+ 801us) server_negotiation.cc:407] Connection header received
0924 00:55:48.392118 (+ 2089us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:48.392124 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:48.392213 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:48.392344 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:55:48.394091 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:48.394915 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:48.396464 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:48.396886 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:48.397064 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:48.400522 (+ 3458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:55:48.400527 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:55:48.400530 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:55:48.400560 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:55:48.403111 (+ 2551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:48.403831 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:48.403837 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:48.403841 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:48.403961 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:48.404396 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:48.404403 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:48.404407 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:48.404673 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:55:48.404822 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:48.405507 (+ 685us) server_negotiation.cc:299] Negotiation successful
0924 00:55:48.405725 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":237,"threads_started":1}
I20260924 00:55:48.407140 8695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:48.389181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:37613 (local address 127.8.64.129:52207)
0924 00:55:48.389863 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0924 00:55:48.389909 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0924 00:55:48.391739 (+ 1830us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:55:48.392607 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:55:48.392616 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:55:48.393197 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:55:48.393913 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:48.393934 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:48.395074 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:48.395079 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:48.396315 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:48.396326 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:48.397121 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:48.397129 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:48.397384 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:48.398092 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:55:48.398114 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:55:48.400297 (+ 2183us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:55:48.403251 (+ 2954us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:55:48.403260 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:55:48.403277 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:55:48.403707 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:55:48.404091 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:55:48.404095 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:55:48.404099 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:55:48.404285 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:55:48.404808 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:55:48.404816 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:55:48.405156 (+ 340us) client_negotiation.cc:769] Sending connection context
0924 00:55:48.405427 (+ 271us) client_negotiation.cc:240] Negotiation successful
0924 00:55:48.405972 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":274,"threads_started":1}
WARNING: no policy specified for kudu/127.8.64.130@KRBTEST.COM; defaulting to no policy
I20260924 00:55:48.408664 8692 heartbeater.cc:347] Connected to a master server at 127.8.64.190:37613
I20260924 00:55:48.409005 8692 heartbeater.cc:464] Registering TS with master...
I20260924 00:55:48.409708 8692 heartbeater.cc:511] Master 127.8.64.190:37613 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.130@KRBTEST.COM" created.
I20260924 00:55:48.411900 8502 ts_manager.cc:194] Registered new tserver with Master: ce7ea4b474f24db0addebb82965fdd6c (127.8.64.129:38999)
I20260924 00:55:48.413301 8502 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.129@KRBTEST.COM'} at 127.8.64.129:52207
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab.
Entry for principal kudu/127.8.64.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab.
I20260924 00:55:48.434688 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.130:0
--local_ip_for_outbound_sockets=127.8.64.130
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:37613
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab
--principal=kudu/127.8.64.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:41613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:55:48.752017 8698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:55:48.752319 8698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:55:48.752425 8698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:55:48.759728 8698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:55:48.759831 8698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:55:48.759945 8698 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.130
I20260924 00:55:48.769361 8698 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:41613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab
--principal=kudu/127.8.64.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:37613
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.130
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:55:48.770946 8698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:55:48.772209 8698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:55:48.784818 8703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:55:48.784874 8706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:55:48.785063 8704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:55:48.787806 8698 server_base.cc:1034] running on GCE node
I20260924 00:55:48.788434 8698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:55:48.789378 8698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:55:48.790573 8698 hybrid_clock.cc:648] HybridClock initialized: now 1790211348790532 us; error 33 us; skew 500 ppm
I20260924 00:55:48.790911 8698 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:55:48 dist-test-slave-mmhs krb5kdc[8457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211348, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:55:48.793872 8698 init.cc:377] Logged in from keytab as kudu/127.8.64.130@KRBTEST.COM (short username kudu)
I20260924 00:55:48.799943 8698 webserver.cc:460] Webserver started at http://127.8.64.130:33751/ using document root <none> and password file <none>
I20260924 00:55:48.800630 8698 fs_manager.cc:359] Metadata directory not provided
I20260924 00:55:48.800755 8698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:55:48.801074 8698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:55:48.804003 8698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-1/data/instance:
uuid: "d633f2ee2279495c88ebc164dfa8f398"
format_stamp: "Formatted at 2026-09-24 00:55:48 on dist-test-slave-mmhs"
server_key: "422fa88842802736e1677f68b6e32432"
server_key_iv: "8cfe545eaf37831ead4c201b3cf46107"
server_key_version: "encryptionkey@0"
I20260924 00:55:48.804750 8698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance:
uuid: "d633f2ee2279495c88ebc164dfa8f398"
format_stamp: "Formatted at 2026-09-24 00:55:48 on dist-test-slave-mmhs"
server_key: "422fa88842802736e1677f68b6e32432"
server_key_iv: "8cfe545eaf37831ead4c201b3cf46107"
server_key_version: "encryptionkey@0"
I20260924 00:55:48.810088 8698 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260924 00:55:48.813287 8713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:55:48.814534 8698 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260924 00:55:48.814682 8698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-1/wal
uuid: "d633f2ee2279495c88ebc164dfa8f398"
format_stamp: "Formatted at 2026-09-24 00:55:48 on dist-test-slave-mmhs"
server_key: "422fa88842802736e1677f68b6e32432"
server_key_iv: "8cfe545eaf37831ead4c201b3cf46107"
server_key_version: "encryptionkey@0"
I20260924 00:55:48.814882 8698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:55:48.850804 8698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:55:48.854743 8698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:55:48.855031 8698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:55:48.856164 8698 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:55:48.857474 8698 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:55:48.857545 8698 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:55:48.857717 8698 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:55:48.857767 8698 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:55:48.893533 8698 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.130:33519
I20260924 00:55:48.893615 8826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.130:33519 every 8 connection(s)
I20260924 00:55:48.894845 8698 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
I20260924 00:55:48.901054 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 8698
I20260924 00:55:48.901279 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance
I20260924 00:55:48.901703 8450 external_mini_cluster.cc:1428] Setting key 680582a268aa0d1ccb4d55429cc90e18
Sep 24 00:55:48 dist-test-slave-mmhs krb5kdc[8457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211348, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.130@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:55:48.918111 8694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:48.897740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:36525 (local address 127.8.64.190:37613)
0924 00:55:48.897929 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:48.897934 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:48.899077 (+ 1143us) server_negotiation.cc:407] Connection header received
0924 00:55:48.900781 (+ 1704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:48.900790 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:48.900883 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:48.901012 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:55:48.902666 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:48.903705 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:48.907064 (+ 3359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:48.907475 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:48.907573 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:48.911396 (+ 3823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:55:48.911401 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:55:48.911404 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:55:48.911433 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:55:48.913881 (+ 2448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:48.914778 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:48.914787 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:48.914795 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:48.914982 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:48.915556 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:48.915561 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:48.915565 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:48.916265 (+ 700us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:55:48.916381 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:48.917308 (+ 927us) server_negotiation.cc:299] Negotiation successful
0924 00:55:48.917606 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260924 00:55:48.919052 8829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:48.898211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:37613 (local address 127.8.64.130:36525)
0924 00:55:48.898920 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0924 00:55:48.898962 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 00:55:48.900529 (+ 1567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:55:48.901228 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:55:48.901241 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:55:48.901814 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:55:48.902465 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:48.902508 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:48.903946 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:48.903953 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:48.906889 (+ 2936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:48.906908 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:48.907740 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:48.907750 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:48.908149 (+ 399us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:48.908937 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:55:48.908964 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:55:48.911163 (+ 2199us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:55:48.914031 (+ 2868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:55:48.914041 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:55:48.914058 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:55:48.914630 (+ 572us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:55:48.915155 (+ 525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:55:48.915159 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:55:48.915162 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:55:48.915407 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:55:48.916462 (+ 1055us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:55:48.916470 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:55:48.916857 (+ 387us) client_negotiation.cc:769] Sending connection context
0924 00:55:48.917138 (+ 281us) client_negotiation.cc:240] Negotiation successful
0924 00:55:48.917643 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":259,"threads_started":1}
I20260924 00:55:48.920701 8827 heartbeater.cc:347] Connected to a master server at 127.8.64.190:37613
I20260924 00:55:48.921013 8827 heartbeater.cc:464] Registering TS with master...
I20260924 00:55:48.921772 8827 heartbeater.cc:511] Master 127.8.64.190:37613 requested a full tablet report, sending...
I20260924 00:55:48.923296 8502 ts_manager.cc:194] Registered new tserver with Master: d633f2ee2279495c88ebc164dfa8f398 (127.8.64.130:33519)
WARNING: no policy specified for kudu/127.8.64.131@KRBTEST.COM; defaulting to no policy
I20260924 00:55:48.924197 8502 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.130@KRBTEST.COM'} at 127.8.64.130:36525
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab.
Entry for principal kudu/127.8.64.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab.
I20260924 00:55:48.947643 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.131:0
--local_ip_for_outbound_sockets=127.8.64.131
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:37613
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab
--principal=kudu/127.8.64.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:41613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:55:49.278155 8832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:55:49.278559 8832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:55:49.278728 8832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:55:49.285853 8832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:55:49.285948 8832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:55:49.286056 8832 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.131
I20260924 00:55:49.296120 8832 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:41613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab
--principal=kudu/127.8.64.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:37613
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.131
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:55:49.297312 8832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:55:49.298617 8832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:55:49.312816 8838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:55:49.312779 8840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:55:49.312855 8837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:55:49.314297 8832 server_base.cc:1034] running on GCE node
I20260924 00:55:49.314986 8832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:55:49.315851 8832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:55:49.317080 8832 hybrid_clock.cc:648] HybridClock initialized: now 1790211349317025 us; error 48 us; skew 500 ppm
I20260924 00:55:49.317386 8832 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:55:49 dist-test-slave-mmhs krb5kdc[8457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211349, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:55:49.320662 8832 init.cc:377] Logged in from keytab as kudu/127.8.64.131@KRBTEST.COM (short username kudu)
I20260924 00:55:49.321895 8832 webserver.cc:460] Webserver started at http://127.8.64.131:42893/ using document root <none> and password file <none>
I20260924 00:55:49.322599 8832 fs_manager.cc:359] Metadata directory not provided
I20260924 00:55:49.322695 8832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:55:49.322993 8832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:55:49.325717 8832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-2/data/instance:
uuid: "1d48ec6d8bad4801aceecf95f09c7bed"
format_stamp: "Formatted at 2026-09-24 00:55:49 on dist-test-slave-mmhs"
server_key: "39cb487f140172354451a5c5add0c978"
server_key_iv: "4ec516cccc3b9956fb1fd0392c2390c1"
server_key_version: "encryptionkey@0"
I20260924 00:55:49.326511 8832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance:
uuid: "1d48ec6d8bad4801aceecf95f09c7bed"
format_stamp: "Formatted at 2026-09-24 00:55:49 on dist-test-slave-mmhs"
server_key: "39cb487f140172354451a5c5add0c978"
server_key_iv: "4ec516cccc3b9956fb1fd0392c2390c1"
server_key_version: "encryptionkey@0"
I20260924 00:55:49.331519 8832 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260924 00:55:49.334792 8847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:55:49.336081 8832 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 00:55:49.336238 8832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-2/wal
uuid: "1d48ec6d8bad4801aceecf95f09c7bed"
format_stamp: "Formatted at 2026-09-24 00:55:49 on dist-test-slave-mmhs"
server_key: "39cb487f140172354451a5c5add0c978"
server_key_iv: "4ec516cccc3b9956fb1fd0392c2390c1"
server_key_version: "encryptionkey@0"
I20260924 00:55:49.336403 8832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:55:49.349561 8832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:55:49.353740 8832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:55:49.354027 8832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:55:49.354972 8832 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:55:49.356304 8832 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:55:49.356374 8832 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:55:49.356544 8832 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:55:49.356597 8832 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:55:49.394330 8832 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.131:38661
I20260924 00:55:49.394377 8960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.131:38661 every 8 connection(s)
I20260924 00:55:49.395617 8832 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
I20260924 00:55:49.404227 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 8832
I20260924 00:55:49.404424 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance
I20260924 00:55:49.404959 8450 external_mini_cluster.cc:1428] Setting key 13e162553e2b581f6e7b8fef87fae352
Sep 24 00:55:49 dist-test-slave-mmhs krb5kdc[8457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211349, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.131@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:55:49.416527 8692 heartbeater.cc:503] Master 127.8.64.190:37613 was elected leader, sending a full tablet report...
I20260924 00:55:49.422838 8694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:49.398030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:35257 (local address 127.8.64.190:37613)
0924 00:55:49.398201 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:49.398205 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:49.399346 (+ 1141us) server_negotiation.cc:407] Connection header received
0924 00:55:49.401122 (+ 1776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:49.401128 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:49.401213 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:49.401322 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:55:49.402834 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:49.403588 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:49.405846 (+ 2258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:49.406342 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:49.406487 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:49.416392 (+ 9905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:55:49.416397 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:55:49.416400 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:55:49.416431 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:55:49.419695 (+ 3264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:49.420493 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:49.420498 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:49.420502 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:49.420651 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:49.421144 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:49.421149 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:49.421153 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:49.421424 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:55:49.421581 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:49.422244 (+ 663us) server_negotiation.cc:299] Negotiation successful
0924 00:55:49.422499 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260924 00:55:49.423748 8963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:49.398533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:37613 (local address 127.8.64.131:35257)
0924 00:55:49.399202 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0924 00:55:49.399238 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 00:55:49.400857 (+ 1619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:55:49.401542 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:55:49.401551 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:55:49.402019 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:55:49.402670 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:49.402688 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:49.403731 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:49.403743 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:49.405671 (+ 1928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:49.405684 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:49.406609 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:49.406616 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:49.406919 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:49.413387 (+ 6468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:55:49.413412 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:55:49.416183 (+ 2771us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:55:49.419830 (+ 3647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:55:49.419841 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:55:49.419861 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:55:49.420373 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:55:49.420766 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:55:49.420771 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:55:49.420775 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:55:49.421025 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:55:49.421556 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:55:49.421564 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:55:49.421900 (+ 336us) client_negotiation.cc:769] Sending connection context
0924 00:55:49.422156 (+ 256us) client_negotiation.cc:240] Negotiation successful
0924 00:55:49.422684 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":255,"threads_started":1}
I20260924 00:55:49.424924 8961 heartbeater.cc:347] Connected to a master server at 127.8.64.190:37613
I20260924 00:55:49.425226 8961 heartbeater.cc:464] Registering TS with master...
I20260924 00:55:49.425896 8961 heartbeater.cc:511] Master 127.8.64.190:37613 requested a full tablet report, sending...
I20260924 00:55:49.427255 8502 ts_manager.cc:194] Registered new tserver with Master: 1d48ec6d8bad4801aceecf95f09c7bed (127.8.64.131:38661)
I20260924 00:55:49.428066 8502 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.131@KRBTEST.COM'} at 127.8.64.131:35257
I20260924 00:55:49.432796 8450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 24 00:55:49 dist-test-slave-mmhs krb5kdc[8457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211349, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 24 00:55:49 dist-test-slave-mmhs krb5kdc[8457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211349, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.64.129@KRBTEST.COM
I20260924 00:55:49.476223 8968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:49.461266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60602 (local address 127.8.64.129:38999)
0924 00:55:49.462118 (+ 852us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:49.462125 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:49.462152 (+ 27us) server_negotiation.cc:407] Connection header received
0924 00:55:49.462213 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:49.462220 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:49.462503 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:49.462655 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:55:49.464334 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:49.465378 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:49.466594 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:49.467129 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:49.467223 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:49.471151 (+ 3928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:55:49.471161 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:55:49.471175 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:55:49.471208 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:55:49.473824 (+ 2616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:49.474487 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:49.474493 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:49.474495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:49.474619 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:49.474981 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:49.474984 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:49.474986 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:49.475290 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:55:49.475532 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:49.475759 (+ 227us) server_negotiation.cc:299] Negotiation successful
0924 00:55:49.475934 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":719,"thread_start_us":566,"threads_started":1}
W20260924 00:55:49.477401 8607 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60602
Sep 24 00:55:49 dist-test-slave-mmhs krb5kdc[8457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 24 00:55:49 dist-test-slave-mmhs krb5kdc[8457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.129@KRBTEST.COM
I20260924 00:55:49.520816 8968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:49.507312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60606 (local address 127.8.64.129:38999)
0924 00:55:49.507565 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:49.507573 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:49.507833 (+ 260us) server_negotiation.cc:407] Connection header received
0924 00:55:49.508069 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:49.508073 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:49.508175 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:49.508286 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:55:49.509647 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:49.510401 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:49.511599 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:49.512011 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:49.512143 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:49.515941 (+ 3798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:55:49.515946 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:55:49.515950 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:55:49.515981 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:55:49.518473 (+ 2492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:49.519157 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:49.519161 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:49.519163 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:49.519276 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:49.519637 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:49.519640 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:49.519642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:49.519878 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:55:49.520051 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:49.520314 (+ 263us) server_negotiation.cc:299] Negotiation successful
0924 00:55:49.520468 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260924 00:55:49.522327 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 8564
I20260924 00:55:49.535838 8968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:49.523198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60618 (local address 127.8.64.129:38999)
0924 00:55:49.523360 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:49.523364 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:49.523726 (+ 362us) server_negotiation.cc:407] Connection header received
0924 00:55:49.523835 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:49.523838 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:49.523890 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:49.523965 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:55:49.525050 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:49.525722 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:49.527248 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:49.527835 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:49.527965 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:49.530172 (+ 2207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:55:49.530177 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:55:49.530180 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:55:49.530210 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:55:49.533166 (+ 2956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:49.533817 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:49.533824 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:49.533827 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:49.534028 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:49.534468 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:49.534475 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:49.534478 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:49.534732 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:55:49.534985 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:49.535272 (+ 287us) server_negotiation.cc:299] Negotiation successful
0924 00:55:49.535474 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260924 00:55:49.536504 8687 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:55:49.618593 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 8564
I20260924 00:55:49.635030 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 8698
Sep 24 00:55:49 dist-test-slave-mmhs krb5kdc[8457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.130@KRBTEST.COM
I20260924 00:55:49.650874 8975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:49.635878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60212 (local address 127.8.64.130:33519)
0924 00:55:49.636415 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:49.636423 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:49.636448 (+ 25us) server_negotiation.cc:407] Connection header received
0924 00:55:49.636505 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:49.636513 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:49.636724 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:49.636847 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:55:49.637994 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:49.639143 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:49.640523 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:49.641144 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:49.641283 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:49.645191 (+ 3908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:55:49.645201 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:55:49.645216 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:55:49.645256 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:55:49.648080 (+ 2824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:49.648747 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:49.648755 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:49.648757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:49.648883 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:49.649282 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:49.649287 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:49.649290 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:49.649617 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:55:49.649924 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:49.650218 (+ 294us) server_negotiation.cc:299] Negotiation successful
0924 00:55:49.650514 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":297,"threads_started":1}
I20260924 00:55:49.652098 8822 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:55:49.732944 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 8698
I20260924 00:55:49.755398 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 8832
Sep 24 00:55:49 dist-test-slave-mmhs krb5kdc[8457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.131@KRBTEST.COM
I20260924 00:55:49.771656 8977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:49.756464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47132 (local address 127.8.64.131:38661)
0924 00:55:49.757187 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:49.757194 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:49.757222 (+ 28us) server_negotiation.cc:407] Connection header received
0924 00:55:49.757281 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:49.757287 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:49.757507 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:49.757674 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:55:49.759060 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:49.760139 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:49.761393 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:49.761931 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:49.762063 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:49.766162 (+ 4099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:55:49.766173 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:55:49.766187 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:55:49.766217 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:55:49.768815 (+ 2598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:49.769504 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:49.769511 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:49.769513 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:49.769627 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:49.770008 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:49.770013 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:49.770018 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:49.770432 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:55:49.770739 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:49.771045 (+ 306us) server_negotiation.cc:299] Negotiation successful
0924 00:55:49.771307 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":423,"threads_started":1}
I20260924 00:55:49.772926 8956 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:55:49.860049 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 8832
I20260924 00:55:49.876236 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 8470
I20260924 00:55:49.877393 8532 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:55:49.978226 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 8470
2026-09-24T00:55:49Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (6846 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:55:50 dist-test-slave-mmhs krb5kdc[8982](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:55:50 dist-test-slave-mmhs krb5kdc[8982](info): set up 2 sockets
Sep 24 00:55:50 dist-test-slave-mmhs krb5kdc[8982](info): commencing operation
krb5kdc: starting...
W20260924 00:55:51.064005 8450 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.000s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Sep 24 00:55:51 dist-test-slave-mmhs krb5kdc[8982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:55:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:55:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.64.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
I20260924 00:55:51.249605 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:44691
--webserver_interface=127.8.64.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:43745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:44691
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:55:51.571594 8995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:55:51.571902 8995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:55:51.572093 8995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:55:51.579288 8995 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:55:51.579392 8995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:55:51.579485 8995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:55:51.579572 8995 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:55:51.579662 8995 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:55:51.589370 8995 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:43745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:44691
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:44691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:55:51.590731 8995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:55:51.592084 8995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:55:51.604135 9000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:55:51.604046 9003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:55:51.604120 9001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:55:51.604871 8995 server_base.cc:1034] running on GCE node
I20260924 00:55:51.605741 8995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:55:51.607173 8995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:55:51.608420 8995 hybrid_clock.cc:648] HybridClock initialized: now 1790211351608373 us; error 36 us; skew 500 ppm
I20260924 00:55:51.608763 8995 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:55:51 dist-test-slave-mmhs krb5kdc[8982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211351, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:55:51.611874 8995 init.cc:377] Logged in from keytab as kudu/127.8.64.190@KRBTEST.COM (short username kudu)
I20260924 00:55:51.613446 8995 webserver.cc:460] Webserver started at http://127.8.64.190:44415/ using document root <none> and password file <none>
I20260924 00:55:51.614144 8995 fs_manager.cc:359] Metadata directory not provided
I20260924 00:55:51.614339 8995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:55:51.614662 8995 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:55:51.617553 8995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "92d80094a0b74372ac6537f7336212ea"
format_stamp: "Formatted at 2026-09-24 00:55:51 on dist-test-slave-mmhs"
server_key: "1cfd1c313372d3c4dcfe36b49b18d700"
server_key_iv: "435e877df4eb26a012de396a92efbcfb"
server_key_version: "encryptionkey@0"
I20260924 00:55:51.618369 8995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "92d80094a0b74372ac6537f7336212ea"
format_stamp: "Formatted at 2026-09-24 00:55:51 on dist-test-slave-mmhs"
server_key: "1cfd1c313372d3c4dcfe36b49b18d700"
server_key_iv: "435e877df4eb26a012de396a92efbcfb"
server_key_version: "encryptionkey@0"
I20260924 00:55:51.623409 8995 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260924 00:55:51.626693 9010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:55:51.627954 8995 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260924 00:55:51.628160 8995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "92d80094a0b74372ac6537f7336212ea"
format_stamp: "Formatted at 2026-09-24 00:55:51 on dist-test-slave-mmhs"
server_key: "1cfd1c313372d3c4dcfe36b49b18d700"
server_key_iv: "435e877df4eb26a012de396a92efbcfb"
server_key_version: "encryptionkey@0"
I20260924 00:55:51.628350 8995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:55:51.657860 8995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:55:51.661979 8995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:55:51.662312 8995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:55:51.680740 8995 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.190:44691
I20260924 00:55:51.680819 9062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.190:44691 every 8 connection(s)
I20260924 00:55:51.682169 8995 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 00:55:51.686043 9063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:55:51.686254 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 8995
I20260924 00:55:51.686524 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 00:55:51.687335 8450 external_mini_cluster.cc:1428] Setting key 36d7361b1958f9eef6d41c9eb132fd2a
I20260924 00:55:51.700033 9063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92d80094a0b74372ac6537f7336212ea: Bootstrap starting.
Sep 24 00:55:51 dist-test-slave-mmhs krb5kdc[8982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:55:51.703569 9063 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92d80094a0b74372ac6537f7336212ea: Neither blocks nor log segments found. Creating new log.
I20260924 00:55:51.704793 9063 log.cc:824] T 00000000000000000000000000000000 P 92d80094a0b74372ac6537f7336212ea: Log is configured to *not* fsync() on all Append() calls
I20260924 00:55:51.707607 9063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92d80094a0b74372ac6537f7336212ea: No bootstrap required, opened a new log
I20260924 00:55:51.710345 9066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:51.689329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42652 (local address 127.8.64.190:44691)
0924 00:55:51.690154 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:51.690165 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:51.690208 (+ 43us) server_negotiation.cc:407] Connection header received
0924 00:55:51.692344 (+ 2136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:51.692369 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:51.692771 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:51.693095 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:55:51.694802 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:51.696074 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:51.697830 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:51.698534 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:51.698658 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:51.703169 (+ 4511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:55:51.703177 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:55:51.703191 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:55:51.703230 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:55:51.706384 (+ 3154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:51.707019 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:51.707035 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:51.707041 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:51.707196 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:51.707573 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:51.707578 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:51.707583 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:51.708142 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:55:51.708439 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:51.708902 (+ 463us) server_negotiation.cc:299] Negotiation successful
0924 00:55:51.709322 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":625,"thread_start_us":385,"threads_started":1}
I20260924 00:55:51.715332 9063 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92d80094a0b74372ac6537f7336212ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92d80094a0b74372ac6537f7336212ea" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44691 } }
I20260924 00:55:51.715777 9063 raft_consensus.cc:374] T 00000000000000000000000000000000 P 92d80094a0b74372ac6537f7336212ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:55:51.715901 9063 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92d80094a0b74372ac6537f7336212ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92d80094a0b74372ac6537f7336212ea, State: Initialized, Role: FOLLOWER
I20260924 00:55:51.716565 9063 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92d80094a0b74372ac6537f7336212ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92d80094a0b74372ac6537f7336212ea" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44691 } }
I20260924 00:55:51.716838 9063 raft_consensus.cc:388] T 00000000000000000000000000000000 P 92d80094a0b74372ac6537f7336212ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:55:51.716974 9063 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92d80094a0b74372ac6537f7336212ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:55:51.717146 9063 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92d80094a0b74372ac6537f7336212ea [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:55:51.719286 9063 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92d80094a0b74372ac6537f7336212ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92d80094a0b74372ac6537f7336212ea" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44691 } }
I20260924 00:55:51.719803 9063 leader_election.cc:302] T 00000000000000000000000000000000 P 92d80094a0b74372ac6537f7336212ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 92d80094a0b74372ac6537f7336212ea; no voters:
I20260924 00:55:51.720428 9063 leader_election.cc:288] T 00000000000000000000000000000000 P 92d80094a0b74372ac6537f7336212ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:55:51.720614 9068 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92d80094a0b74372ac6537f7336212ea [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:55:51.722401 9068 raft_consensus.cc:686] T 00000000000000000000000000000000 P 92d80094a0b74372ac6537f7336212ea [term 1 LEADER]: Becoming Leader. State: Replica: 92d80094a0b74372ac6537f7336212ea, State: Running, Role: LEADER
I20260924 00:55:51.722922 9063 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92d80094a0b74372ac6537f7336212ea [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:55:51.722929 9068 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92d80094a0b74372ac6537f7336212ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92d80094a0b74372ac6537f7336212ea" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44691 } }
I20260924 00:55:51.726071 9070 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92d80094a0b74372ac6537f7336212ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92d80094a0b74372ac6537f7336212ea. Latest consensus state: current_term: 1 leader_uuid: "92d80094a0b74372ac6537f7336212ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92d80094a0b74372ac6537f7336212ea" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44691 } } }
I20260924 00:55:51.726169 9069 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92d80094a0b74372ac6537f7336212ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "92d80094a0b74372ac6537f7336212ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92d80094a0b74372ac6537f7336212ea" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44691 } } }
I20260924 00:55:51.726518 9070 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92d80094a0b74372ac6537f7336212ea [sys.catalog]: This master's current role is: LEADER
I20260924 00:55:51.726585 9069 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92d80094a0b74372ac6537f7336212ea [sys.catalog]: This master's current role is: LEADER
I20260924 00:55:51.728950 9076 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:55:51.734164 9076 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:55:51.743794 9076 catalog_manager.cc:1347] Generated new cluster ID: 7519cf8cf2a4414bab606e1850777975
I20260924 00:55:51.743923 9076 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:55:51.767170 9076 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:55:51.768486 9076 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:55:51.776119 9076 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 92d80094a0b74372ac6537f7336212ea: Generated new TSK 0
I20260924 00:55:51.776923 9076 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.64.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab.
Entry for principal kudu/127.8.64.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab.
I20260924 00:55:51.834206 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.129:0
--local_ip_for_outbound_sockets=127.8.64.129
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:44691
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab
--principal=kudu/127.8.64.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:43745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:55:52.157037 9089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:55:52.157302 9089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:55:52.157449 9089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:55:52.164546 9089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:55:52.164647 9089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:55:52.164845 9089 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.129
I20260924 00:55:52.174362 9089 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:43745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab
--principal=kudu/127.8.64.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:44691
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.129
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:55:52.175586 9089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:55:52.177244 9089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:55:52.189915 9095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:55:52.189977 9097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:55:52.189996 9094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:55:52.190758 9089 server_base.cc:1034] running on GCE node
I20260924 00:55:52.191325 9089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:55:52.192219 9089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:55:52.193446 9089 hybrid_clock.cc:648] HybridClock initialized: now 1790211352193389 us; error 64 us; skew 500 ppm
I20260924 00:55:52.193743 9089 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:55:52 dist-test-slave-mmhs krb5kdc[8982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211352, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:55:52.196765 9089 init.cc:377] Logged in from keytab as kudu/127.8.64.129@KRBTEST.COM (short username kudu)
I20260924 00:55:52.198058 9089 webserver.cc:460] Webserver started at http://127.8.64.129:33823/ using document root <none> and password file <none>
I20260924 00:55:52.198753 9089 fs_manager.cc:359] Metadata directory not provided
I20260924 00:55:52.198846 9089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:55:52.199087 9089 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:55:52.201817 9089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-0/data/instance:
uuid: "c106e816b92e49e6bf085bf0f4758cbb"
format_stamp: "Formatted at 2026-09-24 00:55:52 on dist-test-slave-mmhs"
server_key: "af61943b6dd38a39ebe3f70f3c085427"
server_key_iv: "fb3b920232ac8f0242d265a481317b5f"
server_key_version: "encryptionkey@0"
I20260924 00:55:52.202577 9089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance:
uuid: "c106e816b92e49e6bf085bf0f4758cbb"
format_stamp: "Formatted at 2026-09-24 00:55:52 on dist-test-slave-mmhs"
server_key: "af61943b6dd38a39ebe3f70f3c085427"
server_key_iv: "fb3b920232ac8f0242d265a481317b5f"
server_key_version: "encryptionkey@0"
I20260924 00:55:52.207340 9089 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260924 00:55:52.210865 9104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:55:52.212214 9089 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260924 00:55:52.212376 9089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-0/wal
uuid: "c106e816b92e49e6bf085bf0f4758cbb"
format_stamp: "Formatted at 2026-09-24 00:55:52 on dist-test-slave-mmhs"
server_key: "af61943b6dd38a39ebe3f70f3c085427"
server_key_iv: "fb3b920232ac8f0242d265a481317b5f"
server_key_version: "encryptionkey@0"
I20260924 00:55:52.212558 9089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:55:52.238561 9089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:55:52.242636 9089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:55:52.242913 9089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:55:52.243849 9089 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:55:52.245106 9089 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:55:52.245177 9089 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:55:52.245318 9089 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:55:52.245381 9089 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:55:52.278578 9089 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.129:40955
I20260924 00:55:52.278647 9217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.129:40955 every 8 connection(s)
I20260924 00:55:52.279815 9089 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
I20260924 00:55:52.282914 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 9089
I20260924 00:55:52.283133 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance
I20260924 00:55:52.283950 8450 external_mini_cluster.cc:1428] Setting key 854bbe1147f9a013c1c9dd2516227e0d
Sep 24 00:55:52 dist-test-slave-mmhs krb5kdc[8982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211352, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.64.130@KRBTEST.COM; defaulting to no policy
I20260924 00:55:52.299394 9220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:52.282359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:42937 (local address 127.8.64.190:44691)
0924 00:55:52.282808 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:52.282812 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:52.283907 (+ 1095us) server_negotiation.cc:407] Connection header received
0924 00:55:52.285717 (+ 1810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:52.285723 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:52.285802 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:52.285936 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:55:52.287442 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:52.288171 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:52.289701 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:52.290105 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:52.290254 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:52.293627 (+ 3373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:55:52.293631 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:55:52.293634 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:55:52.293664 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:55:52.296301 (+ 2637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:52.297058 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:52.297071 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:52.297074 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:52.297184 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:52.297628 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:52.297633 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:52.297637 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:52.297889 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:55:52.298046 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:52.298799 (+ 753us) server_negotiation.cc:299] Negotiation successful
0924 00:55:52.299020 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":245,"threads_started":1}
I20260924 00:55:52.300403 9221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:52.282980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44691 (local address 127.8.64.129:42937)
0924 00:55:52.283745 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0924 00:55:52.283788 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 00:55:52.285467 (+ 1679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:55:52.286160 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:55:52.286168 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:55:52.286742 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:55:52.287278 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:52.287294 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:52.288317 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:52.288329 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:52.289547 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:52.289557 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:52.290357 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:52.290367 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:52.290607 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:52.291295 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:55:52.291316 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:55:52.293399 (+ 2083us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:55:52.296466 (+ 3067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:55:52.296475 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:55:52.296494 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:55:52.296920 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:55:52.297321 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:55:52.297326 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:55:52.297329 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:55:52.297511 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:55:52.298032 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:55:52.298040 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:55:52.298431 (+ 391us) client_negotiation.cc:769] Sending connection context
0924 00:55:52.298694 (+ 263us) client_negotiation.cc:240] Negotiation successful
0924 00:55:52.299233 (+ 539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":323,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.130@KRBTEST.COM" created.
I20260924 00:55:52.302122 9218 heartbeater.cc:347] Connected to a master server at 127.8.64.190:44691
I20260924 00:55:52.302493 9218 heartbeater.cc:464] Registering TS with master...
I20260924 00:55:52.303196 9218 heartbeater.cc:511] Master 127.8.64.190:44691 requested a full tablet report, sending...
I20260924 00:55:52.305275 9027 ts_manager.cc:194] Registered new tserver with Master: c106e816b92e49e6bf085bf0f4758cbb (127.8.64.129:40955)
I20260924 00:55:52.306807 9027 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.129@KRBTEST.COM'} at 127.8.64.129:42937
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab.
Entry for principal kudu/127.8.64.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab.
I20260924 00:55:52.324121 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.130:0
--local_ip_for_outbound_sockets=127.8.64.130
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:44691
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab
--principal=kudu/127.8.64.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:43745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:55:52.667819 9224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:55:52.668107 9224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:55:52.668200 9224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:55:52.675454 9224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:55:52.675589 9224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:55:52.675705 9224 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.130
I20260924 00:55:52.685442 9224 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:43745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab
--principal=kudu/127.8.64.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:44691
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.130
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:55:52.686677 9224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:55:52.687990 9224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:55:52.701077 9232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:55:52.701104 9230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:55:52.701269 9229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:55:52.701749 9224 server_base.cc:1034] running on GCE node
I20260924 00:55:52.702565 9224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:55:52.703564 9224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:55:52.704828 9224 hybrid_clock.cc:648] HybridClock initialized: now 1790211352704773 us; error 65 us; skew 500 ppm
I20260924 00:55:52.705209 9224 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:55:52 dist-test-slave-mmhs krb5kdc[8982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211352, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:55:52.708496 9224 init.cc:377] Logged in from keytab as kudu/127.8.64.130@KRBTEST.COM (short username kudu)
I20260924 00:55:52.709961 9224 webserver.cc:460] Webserver started at http://127.8.64.130:36811/ using document root <none> and password file <none>
I20260924 00:55:52.710741 9224 fs_manager.cc:359] Metadata directory not provided
I20260924 00:55:52.710843 9224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:55:52.711164 9224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:55:52.714250 9224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-1/data/instance:
uuid: "7bc362f1de504681a8efec0e6417abc9"
format_stamp: "Formatted at 2026-09-24 00:55:52 on dist-test-slave-mmhs"
server_key: "5a49697ecded426f0f7025e3bc3c08f7"
server_key_iv: "9d5915d633702268ac1eec3d55df30cf"
server_key_version: "encryptionkey@0"
I20260924 00:55:52.715139 9224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance:
uuid: "7bc362f1de504681a8efec0e6417abc9"
format_stamp: "Formatted at 2026-09-24 00:55:52 on dist-test-slave-mmhs"
server_key: "5a49697ecded426f0f7025e3bc3c08f7"
server_key_iv: "9d5915d633702268ac1eec3d55df30cf"
server_key_version: "encryptionkey@0"
I20260924 00:55:52.720419 9224 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260924 00:55:52.724110 9239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:55:52.725649 9224 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260924 00:55:52.725835 9224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-1/wal
uuid: "7bc362f1de504681a8efec0e6417abc9"
format_stamp: "Formatted at 2026-09-24 00:55:52 on dist-test-slave-mmhs"
server_key: "5a49697ecded426f0f7025e3bc3c08f7"
server_key_iv: "9d5915d633702268ac1eec3d55df30cf"
server_key_version: "encryptionkey@0"
I20260924 00:55:52.726037 9224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:55:52.752477 9224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:55:52.757004 9224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:55:52.757324 9224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:55:52.758412 9224 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:55:52.759820 9224 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:55:52.759927 9224 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:55:52.760071 9224 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:55:52.760144 9224 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:55:52.794440 9224 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.130:33461
I20260924 00:55:52.794524 9352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.130:33461 every 8 connection(s)
I20260924 00:55:52.795768 9224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
I20260924 00:55:52.797271 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 9224
I20260924 00:55:52.797468 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance
I20260924 00:55:52.797996 8450 external_mini_cluster.cc:1428] Setting key 70634354e7c76845255a0fc9961622dd
Sep 24 00:55:52 dist-test-slave-mmhs krb5kdc[8982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211352, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.130@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.64.131@KRBTEST.COM; defaulting to no policy
I20260924 00:55:52.819223 9220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:52.798644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:35101 (local address 127.8.64.190:44691)
0924 00:55:52.798863 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:52.798869 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:52.800400 (+ 1531us) server_negotiation.cc:407] Connection header received
0924 00:55:52.802410 (+ 2010us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:52.802415 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:52.802499 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:52.802626 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:55:52.804176 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:52.804867 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:52.806560 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:52.806988 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:52.807097 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:52.811664 (+ 4567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:55:52.811673 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:55:52.811681 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:55:52.811726 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:55:52.815669 (+ 3943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:52.816631 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:52.816640 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:52.816646 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:52.816785 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:52.817392 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:52.817396 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:52.817398 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:52.817646 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:55:52.817755 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:52.818704 (+ 949us) server_negotiation.cc:299] Negotiation successful
0924 00:55:52.818923 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.131@KRBTEST.COM" created.
I20260924 00:55:52.820366 9355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:52.799222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44691 (local address 127.8.64.130:35101)
0924 00:55:52.800249 (+ 1027us) negotiation.cc:107] Waiting for socket to connect
0924 00:55:52.800294 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0924 00:55:52.802102 (+ 1808us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:55:52.802879 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:55:52.802888 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:55:52.803406 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:55:52.803988 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:52.804006 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:52.805031 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:52.805037 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:52.806384 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:52.806401 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:52.807220 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:52.807229 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:52.807561 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:52.808310 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:55:52.808344 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:55:52.811401 (+ 3057us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:55:52.815874 (+ 4473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:55:52.815881 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:55:52.815895 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:55:52.816479 (+ 584us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:55:52.816954 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:55:52.816959 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:55:52.816969 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:55:52.817257 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:55:52.817755 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:55:52.817762 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:55:52.818189 (+ 427us) client_negotiation.cc:769] Sending connection context
0924 00:55:52.818521 (+ 332us) client_negotiation.cc:240] Negotiation successful
0924 00:55:52.818918 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":755,"thread_start_us":531,"threads_started":1}
I20260924 00:55:52.821947 9353 heartbeater.cc:347] Connected to a master server at 127.8.64.190:44691
I20260924 00:55:52.822270 9353 heartbeater.cc:464] Registering TS with master...
I20260924 00:55:52.823042 9353 heartbeater.cc:511] Master 127.8.64.190:44691 requested a full tablet report, sending...
I20260924 00:55:52.824697 9027 ts_manager.cc:194] Registered new tserver with Master: 7bc362f1de504681a8efec0e6417abc9 (127.8.64.130:33461)
I20260924 00:55:52.825826 9027 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.130@KRBTEST.COM'} at 127.8.64.130:35101
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab.
Entry for principal kudu/127.8.64.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab.
I20260924 00:55:52.845513 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.131:0
--local_ip_for_outbound_sockets=127.8.64.131
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:44691
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab
--principal=kudu/127.8.64.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:43745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:55:53.162560 9358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:55:53.162840 9358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:55:53.162945 9358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:55:53.170190 9358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:55:53.170372 9358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:55:53.170521 9358 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.131
I20260924 00:55:53.179978 9358 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:43745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab
--principal=kudu/127.8.64.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:44691
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.131
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:55:53.181270 9358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:55:53.182533 9358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:55:53.197464 9363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:55:53.197556 9364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:55:53.197618 9366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:55:53.198525 9358 server_base.cc:1034] running on GCE node
I20260924 00:55:53.199110 9358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:55:53.199937 9358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:55:53.201128 9358 hybrid_clock.cc:648] HybridClock initialized: now 1790211353201091 us; error 25 us; skew 500 ppm
I20260924 00:55:53.201390 9358 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:55:53 dist-test-slave-mmhs krb5kdc[8982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211353, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:55:53.204339 9358 init.cc:377] Logged in from keytab as kudu/127.8.64.131@KRBTEST.COM (short username kudu)
I20260924 00:55:53.205752 9358 webserver.cc:460] Webserver started at http://127.8.64.131:34619/ using document root <none> and password file <none>
I20260924 00:55:53.206468 9358 fs_manager.cc:359] Metadata directory not provided
I20260924 00:55:53.206594 9358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:55:53.206902 9358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:55:53.209632 9358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-2/data/instance:
uuid: "4c95c6fcc1ff4406b81c0d791714860a"
format_stamp: "Formatted at 2026-09-24 00:55:53 on dist-test-slave-mmhs"
server_key: "e73d8bd6828cf19dfdd3eb75ea430f63"
server_key_iv: "df00cf253dcf4f12c334b515cd8d248b"
server_key_version: "encryptionkey@0"
I20260924 00:55:53.210460 9358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance:
uuid: "4c95c6fcc1ff4406b81c0d791714860a"
format_stamp: "Formatted at 2026-09-24 00:55:53 on dist-test-slave-mmhs"
server_key: "e73d8bd6828cf19dfdd3eb75ea430f63"
server_key_iv: "df00cf253dcf4f12c334b515cd8d248b"
server_key_version: "encryptionkey@0"
I20260924 00:55:53.215476 9358 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260924 00:55:53.218830 9373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:55:53.220163 9358 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260924 00:55:53.220284 9358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-2/wal
uuid: "4c95c6fcc1ff4406b81c0d791714860a"
format_stamp: "Formatted at 2026-09-24 00:55:53 on dist-test-slave-mmhs"
server_key: "e73d8bd6828cf19dfdd3eb75ea430f63"
server_key_iv: "df00cf253dcf4f12c334b515cd8d248b"
server_key_version: "encryptionkey@0"
I20260924 00:55:53.220433 9358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:55:53.244189 9358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:55:53.248200 9358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:55:53.248430 9358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:55:53.249300 9358 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:55:53.250562 9358 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:55:53.250633 9358 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:55:53.250712 9358 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:55:53.250756 9358 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:55:53.283525 9358 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.131:37109
I20260924 00:55:53.283571 9485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.131:37109 every 8 connection(s)
I20260924 00:55:53.284718 9358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
I20260924 00:55:53.292676 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 9358
I20260924 00:55:53.292879 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance
I20260924 00:55:53.293332 8450 external_mini_cluster.cc:1428] Setting key cd17a1fca8a6dbb7d7f9c15fc0692549
Sep 24 00:55:53 dist-test-slave-mmhs krb5kdc[8982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211353, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.131@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:55:53.304157 9220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:53.287251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:32935 (local address 127.8.64.190:44691)
0924 00:55:53.287431 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:53.287435 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:53.288742 (+ 1307us) server_negotiation.cc:407] Connection header received
0924 00:55:53.290530 (+ 1788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:53.290535 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:53.290689 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:53.290802 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:55:53.292276 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:53.293167 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:53.295153 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:53.295664 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:53.295805 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:53.299242 (+ 3437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:55:53.299246 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:55:53.299249 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:55:53.299279 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:55:53.301628 (+ 2349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:53.302266 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:53.302269 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:53.302272 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:53.302450 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:53.302799 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:53.302803 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:53.302805 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:53.303054 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:55:53.303159 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:53.303754 (+ 595us) server_negotiation.cc:299] Negotiation successful
0924 00:55:53.303900 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260924 00:55:53.305048 9488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:53.287676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44691 (local address 127.8.64.131:32935)
0924 00:55:53.288566 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0924 00:55:53.288607 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 00:55:53.290199 (+ 1592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:55:53.291049 (+ 850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:55:53.291057 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:55:53.291555 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:55:53.292106 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:53.292122 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:53.293320 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:53.293329 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:53.295012 (+ 1683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:53.295027 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:53.295890 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:53.295900 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:53.296198 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:53.296860 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:55:53.296882 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:55:53.299054 (+ 2172us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:55:53.301743 (+ 2689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:55:53.301750 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:55:53.301761 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:55:53.302162 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:55:53.302545 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:55:53.302548 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:55:53.302550 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:55:53.302702 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:55:53.303168 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:55:53.303173 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:55:53.303441 (+ 268us) client_negotiation.cc:769] Sending connection context
0924 00:55:53.303656 (+ 215us) client_negotiation.cc:240] Negotiation successful
0924 00:55:53.304000 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":288,"threads_started":1}
I20260924 00:55:53.306463 9486 heartbeater.cc:347] Connected to a master server at 127.8.64.190:44691
I20260924 00:55:53.306767 9486 heartbeater.cc:464] Registering TS with master...
I20260924 00:55:53.307420 9486 heartbeater.cc:511] Master 127.8.64.190:44691 requested a full tablet report, sending...
I20260924 00:55:53.308745 9027 ts_manager.cc:194] Registered new tserver with Master: 4c95c6fcc1ff4406b81c0d791714860a (127.8.64.131:37109)
I20260924 00:55:53.309489 9027 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.131@KRBTEST.COM'} at 127.8.64.131:32935
I20260924 00:55:53.309715 9218 heartbeater.cc:503] Master 127.8.64.190:44691 was elected leader, sending a full tablet report...
I20260924 00:55:53.320561 8450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 00:55:53.340281 9220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:53.328271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42662 (local address 127.8.64.190:44691)
0924 00:55:53.328465 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:53.328471 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:53.328942 (+ 471us) server_negotiation.cc:407] Connection header received
0924 00:55:53.329067 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:53.329071 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:53.329128 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:53.329211 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:55:53.330804 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:53.331616 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:53.332901 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:53.333363 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:53.333458 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:53.335671 (+ 2213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:55:53.335675 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:55:53.335678 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:55:53.335704 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:55:53.338059 (+ 2355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:53.338696 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:53.338700 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:53.338702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:53.338820 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:53.339141 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:53.339145 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:53.339147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:53.339439 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:55:53.339618 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:53.339903 (+ 285us) server_negotiation.cc:299] Negotiation successful
0924 00:55:53.340051 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260924 00:55:53.346364 9027 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42662:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:55:53.353493 9027 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:55:53.390765 9499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:53.365056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49346 (local address 127.8.64.130:33461)
0924 00:55:53.382700 (+ 17644us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:53.382716 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:53.382745 (+ 29us) server_negotiation.cc:407] Connection header received
0924 00:55:53.382848 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:53.382857 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:53.383112 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:53.383310 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:55:53.384199 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:53.385674 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:53.388387 (+ 2713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:53.389657 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:53.389811 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:53.390025 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:53.390292 (+ 267us) server_negotiation.cc:299] Negotiation successful
0924 00:55:53.390472 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17492,"thread_start_us":17492,"threads_started":1}
I20260924 00:55:53.390868 9498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:53.364839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.130:33461 (local address 127.0.0.1:49346)
0924 00:55:53.380548 (+ 15709us) negotiation.cc:107] Waiting for socket to connect
0924 00:55:53.380602 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0924 00:55:53.380898 (+ 296us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:55:53.383445 (+ 2547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:55:53.383449 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:55:53.383489 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:55:53.383982 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:53.384000 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:53.385808 (+ 1808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:53.385816 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:53.388189 (+ 2373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:53.388204 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:53.389858 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:53.389864 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:53.390047 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:53.390102 (+ 55us) client_negotiation.cc:769] Sending connection context
0924 00:55:53.390254 (+ 152us) client_negotiation.cc:240] Negotiation successful
0924 00:55:53.390457 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15371,"mutex_wait_us":1,"spinlock_wait_cycles":8064,"thread_start_us":16290,"threads_started":1}
I20260924 00:55:53.391955 9500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:53.365581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:40955 (local address 127.0.0.1:38612)
0924 00:55:53.383156 (+ 17575us) negotiation.cc:107] Waiting for socket to connect
0924 00:55:53.383178 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0924 00:55:53.383305 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:55:53.383899 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:55:53.383904 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:55:53.383934 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:55:53.384321 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:53.384334 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:53.386195 (+ 1861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:53.386200 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:53.388886 (+ 2686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:53.388896 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:53.391186 (+ 2290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:53.391196 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:53.391366 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:53.391396 (+ 30us) client_negotiation.cc:769] Sending connection context
0924 00:55:53.391490 (+ 94us) client_negotiation.cc:240] Negotiation successful
0924 00:55:53.391617 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17427,"thread_start_us":15222,"threads_started":1}
I20260924 00:55:53.392261 9503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:53.366359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38612 (local address 127.8.64.129:40955)
0924 00:55:53.374904 (+ 8545us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:53.374918 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:53.383309 (+ 8391us) server_negotiation.cc:407] Connection header received
0924 00:55:53.383421 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:53.383431 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:53.383670 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:53.383912 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:55:53.384502 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:53.386035 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:53.389055 (+ 3020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:53.390526 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:53.390634 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:53.391288 (+ 654us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:53.391554 (+ 266us) server_negotiation.cc:299] Negotiation successful
0924 00:55:53.391722 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8338,"thread_start_us":8187,"threads_started":1}
I20260924 00:55:53.392846 9502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:53.365744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59216 (local address 127.8.64.131:37109)
0924 00:55:53.382186 (+ 16442us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:53.382198 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:53.382242 (+ 44us) server_negotiation.cc:407] Connection header received
0924 00:55:53.382395 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:53.382408 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:53.382749 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:53.383566 (+ 817us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:55:53.383896 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:53.385500 (+ 1604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:53.388903 (+ 3403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:53.391102 (+ 2199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:53.392049 (+ 947us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:53.392277 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:53.392353 (+ 76us) server_negotiation.cc:299] Negotiation successful
0924 00:55:53.392510 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16234,"thread_start_us":16084,"threads_started":1}
I20260924 00:55:53.393260 9501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:53.365038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.131:37109 (local address 127.0.0.1:59216)
0924 00:55:53.380550 (+ 15512us) negotiation.cc:107] Waiting for socket to connect
0924 00:55:53.380603 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0924 00:55:53.381577 (+ 974us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:55:53.383044 (+ 1467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:55:53.383052 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:55:53.383098 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:55:53.383506 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:53.383519 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:53.385646 (+ 2127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:53.385652 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:53.388701 (+ 3049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:53.388717 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:53.391388 (+ 2671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:53.391393 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:53.391554 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:53.391587 (+ 33us) client_negotiation.cc:769] Sending connection context
0924 00:55:53.391652 (+ 65us) client_negotiation.cc:240] Negotiation successful
0924 00:55:53.391768 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15299,"spinlock_wait_cycles":1715328,"thread_start_us":17829,"threads_started":1}
I20260924 00:55:53.394870 9287 tablet_service.cc:1467] Processing CreateTablet for tablet 05b58b9b730e4a449513d8e29256de0f (DEFAULT_TABLE table=test-table [id=33c9b77b9d2d4ff4a5e820fa850821ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:55:53.395733 9152 tablet_service.cc:1467] Processing CreateTablet for tablet 05b58b9b730e4a449513d8e29256de0f (DEFAULT_TABLE table=test-table [id=33c9b77b9d2d4ff4a5e820fa850821ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:55:53.396350 9287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05b58b9b730e4a449513d8e29256de0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:55:53.396863 9152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05b58b9b730e4a449513d8e29256de0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:55:53.397979 9421 tablet_service.cc:1467] Processing CreateTablet for tablet 05b58b9b730e4a449513d8e29256de0f (DEFAULT_TABLE table=test-table [id=33c9b77b9d2d4ff4a5e820fa850821ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:55:53.399303 9421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05b58b9b730e4a449513d8e29256de0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:55:53.408078 9504 tablet_bootstrap.cc:492] T 05b58b9b730e4a449513d8e29256de0f P c106e816b92e49e6bf085bf0f4758cbb: Bootstrap starting.
I20260924 00:55:53.411465 9504 tablet_bootstrap.cc:654] T 05b58b9b730e4a449513d8e29256de0f P c106e816b92e49e6bf085bf0f4758cbb: Neither blocks nor log segments found. Creating new log.
I20260924 00:55:53.412150 9505 tablet_bootstrap.cc:492] T 05b58b9b730e4a449513d8e29256de0f P 7bc362f1de504681a8efec0e6417abc9: Bootstrap starting.
I20260924 00:55:53.413304 9504 log.cc:824] T 05b58b9b730e4a449513d8e29256de0f P c106e816b92e49e6bf085bf0f4758cbb: Log is configured to *not* fsync() on all Append() calls
I20260924 00:55:53.415634 9506 tablet_bootstrap.cc:492] T 05b58b9b730e4a449513d8e29256de0f P 4c95c6fcc1ff4406b81c0d791714860a: Bootstrap starting.
I20260924 00:55:53.415804 9505 tablet_bootstrap.cc:654] T 05b58b9b730e4a449513d8e29256de0f P 7bc362f1de504681a8efec0e6417abc9: Neither blocks nor log segments found. Creating new log.
I20260924 00:55:53.417002 9505 log.cc:824] T 05b58b9b730e4a449513d8e29256de0f P 7bc362f1de504681a8efec0e6417abc9: Log is configured to *not* fsync() on all Append() calls
I20260924 00:55:53.417783 9504 tablet_bootstrap.cc:492] T 05b58b9b730e4a449513d8e29256de0f P c106e816b92e49e6bf085bf0f4758cbb: No bootstrap required, opened a new log
I20260924 00:55:53.418115 9504 ts_tablet_manager.cc:1397] T 05b58b9b730e4a449513d8e29256de0f P c106e816b92e49e6bf085bf0f4758cbb: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260924 00:55:53.419314 9506 tablet_bootstrap.cc:654] T 05b58b9b730e4a449513d8e29256de0f P 4c95c6fcc1ff4406b81c0d791714860a: Neither blocks nor log segments found. Creating new log.
I20260924 00:55:53.420433 9506 log.cc:824] T 05b58b9b730e4a449513d8e29256de0f P 4c95c6fcc1ff4406b81c0d791714860a: Log is configured to *not* fsync() on all Append() calls
I20260924 00:55:53.423590 9505 tablet_bootstrap.cc:492] T 05b58b9b730e4a449513d8e29256de0f P 7bc362f1de504681a8efec0e6417abc9: No bootstrap required, opened a new log
I20260924 00:55:53.423679 9506 tablet_bootstrap.cc:492] T 05b58b9b730e4a449513d8e29256de0f P 4c95c6fcc1ff4406b81c0d791714860a: No bootstrap required, opened a new log
I20260924 00:55:53.423894 9505 ts_tablet_manager.cc:1397] T 05b58b9b730e4a449513d8e29256de0f P 7bc362f1de504681a8efec0e6417abc9: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.009s
I20260924 00:55:53.423962 9506 ts_tablet_manager.cc:1397] T 05b58b9b730e4a449513d8e29256de0f P 4c95c6fcc1ff4406b81c0d791714860a: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260924 00:55:53.429580 9504 raft_consensus.cc:348] T 05b58b9b730e4a449513d8e29256de0f P c106e816b92e49e6bf085bf0f4758cbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc362f1de504681a8efec0e6417abc9" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 33461 } } peers { permanent_uuid: "4c95c6fcc1ff4406b81c0d791714860a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 37109 } } peers { permanent_uuid: "c106e816b92e49e6bf085bf0f4758cbb" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 40955 } }
I20260924 00:55:53.430124 9504 raft_consensus.cc:374] T 05b58b9b730e4a449513d8e29256de0f P c106e816b92e49e6bf085bf0f4758cbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:55:53.430230 9504 raft_consensus.cc:729] T 05b58b9b730e4a449513d8e29256de0f P c106e816b92e49e6bf085bf0f4758cbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c106e816b92e49e6bf085bf0f4758cbb, State: Initialized, Role: FOLLOWER
I20260924 00:55:53.430938 9504 consensus_queue.cc:260] T 05b58b9b730e4a449513d8e29256de0f P c106e816b92e49e6bf085bf0f4758cbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc362f1de504681a8efec0e6417abc9" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 33461 } } peers { permanent_uuid: "4c95c6fcc1ff4406b81c0d791714860a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 37109 } } peers { permanent_uuid: "c106e816b92e49e6bf085bf0f4758cbb" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 40955 } }
I20260924 00:55:53.432488 9505 raft_consensus.cc:348] T 05b58b9b730e4a449513d8e29256de0f P 7bc362f1de504681a8efec0e6417abc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc362f1de504681a8efec0e6417abc9" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 33461 } } peers { permanent_uuid: "4c95c6fcc1ff4406b81c0d791714860a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 37109 } } peers { permanent_uuid: "c106e816b92e49e6bf085bf0f4758cbb" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 40955 } }
I20260924 00:55:53.432919 9505 raft_consensus.cc:374] T 05b58b9b730e4a449513d8e29256de0f P 7bc362f1de504681a8efec0e6417abc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:55:53.433017 9505 raft_consensus.cc:729] T 05b58b9b730e4a449513d8e29256de0f P 7bc362f1de504681a8efec0e6417abc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc362f1de504681a8efec0e6417abc9, State: Initialized, Role: FOLLOWER
I20260924 00:55:53.433589 9505 consensus_queue.cc:260] T 05b58b9b730e4a449513d8e29256de0f P 7bc362f1de504681a8efec0e6417abc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc362f1de504681a8efec0e6417abc9" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 33461 } } peers { permanent_uuid: "4c95c6fcc1ff4406b81c0d791714860a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 37109 } } peers { permanent_uuid: "c106e816b92e49e6bf085bf0f4758cbb" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 40955 } }
I20260924 00:55:53.434713 9353 heartbeater.cc:503] Master 127.8.64.190:44691 was elected leader, sending a full tablet report...
I20260924 00:55:53.435217 9505 ts_tablet_manager.cc:1428] T 05b58b9b730e4a449513d8e29256de0f P 7bc362f1de504681a8efec0e6417abc9: Time spent starting tablet: real 0.011s user 0.009s sys 0.002s
I20260924 00:55:53.438649 9506 raft_consensus.cc:348] T 05b58b9b730e4a449513d8e29256de0f P 4c95c6fcc1ff4406b81c0d791714860a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc362f1de504681a8efec0e6417abc9" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 33461 } } peers { permanent_uuid: "4c95c6fcc1ff4406b81c0d791714860a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 37109 } } peers { permanent_uuid: "c106e816b92e49e6bf085bf0f4758cbb" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 40955 } }
I20260924 00:55:53.439265 9506 raft_consensus.cc:374] T 05b58b9b730e4a449513d8e29256de0f P 4c95c6fcc1ff4406b81c0d791714860a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:55:53.439389 9506 raft_consensus.cc:729] T 05b58b9b730e4a449513d8e29256de0f P 4c95c6fcc1ff4406b81c0d791714860a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c95c6fcc1ff4406b81c0d791714860a, State: Initialized, Role: FOLLOWER
I20260924 00:55:53.440122 9506 consensus_queue.cc:260] T 05b58b9b730e4a449513d8e29256de0f P 4c95c6fcc1ff4406b81c0d791714860a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc362f1de504681a8efec0e6417abc9" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 33461 } } peers { permanent_uuid: "4c95c6fcc1ff4406b81c0d791714860a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 37109 } } peers { permanent_uuid: "c106e816b92e49e6bf085bf0f4758cbb" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 40955 } }
I20260924 00:55:53.441227 9486 heartbeater.cc:503] Master 127.8.64.190:44691 was elected leader, sending a full tablet report...
I20260924 00:55:53.442065 9504 ts_tablet_manager.cc:1428] T 05b58b9b730e4a449513d8e29256de0f P c106e816b92e49e6bf085bf0f4758cbb: Time spent starting tablet: real 0.024s user 0.011s sys 0.012s
I20260924 00:55:53.442199 9506 ts_tablet_manager.cc:1428] T 05b58b9b730e4a449513d8e29256de0f P 4c95c6fcc1ff4406b81c0d791714860a: Time spent starting tablet: real 0.018s user 0.015s sys 0.000s
W20260924 00:55:53.532184 9219 tablet.cc:2369] T 05b58b9b730e4a449513d8e29256de0f P c106e816b92e49e6bf085bf0f4758cbb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 00:55:53.536230 9487 tablet.cc:2369] T 05b58b9b730e4a449513d8e29256de0f P 4c95c6fcc1ff4406b81c0d791714860a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 00:55:53.547823 9354 tablet.cc:2369] T 05b58b9b730e4a449513d8e29256de0f P 7bc362f1de504681a8efec0e6417abc9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:55:53.589219 9512 raft_consensus.cc:482] T 05b58b9b730e4a449513d8e29256de0f P 7bc362f1de504681a8efec0e6417abc9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:55:53.589566 9512 raft_consensus.cc:504] T 05b58b9b730e4a449513d8e29256de0f P 7bc362f1de504681a8efec0e6417abc9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc362f1de504681a8efec0e6417abc9" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 33461 } } peers { permanent_uuid: "4c95c6fcc1ff4406b81c0d791714860a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 37109 } } peers { permanent_uuid: "c106e816b92e49e6bf085bf0f4758cbb" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 40955 } }
I20260924 00:55:53.591374 9512 leader_election.cc:288] T 05b58b9b730e4a449513d8e29256de0f P 7bc362f1de504681a8efec0e6417abc9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c95c6fcc1ff4406b81c0d791714860a (127.8.64.131:37109), c106e816b92e49e6bf085bf0f4758cbb (127.8.64.129:40955)
I20260924 00:55:53.598703 9502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:53.591799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:60573 (local address 127.8.64.131:37109)
0924 00:55:53.592057 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:53.592065 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:53.593291 (+ 1226us) server_negotiation.cc:407] Connection header received
0924 00:55:53.593440 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:53.593445 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:53.593534 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:53.593628 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:55:53.594243 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:53.595090 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:53.596941 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:53.597665 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:53.597817 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:53.597903 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:53.598283 (+ 380us) server_negotiation.cc:299] Negotiation successful
0924 00:55:53.598431 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260924 00:55:53.598831 9503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:53.591882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:36461 (local address 127.8.64.129:40955)
0924 00:55:53.592087 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:53.592093 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:53.593473 (+ 1380us) server_negotiation.cc:407] Connection header received
0924 00:55:53.593621 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:53.593626 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:53.593701 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:53.593804 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:55:53.594428 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:53.595134 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:53.596906 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:53.597682 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:53.597868 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:53.597943 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:53.598336 (+ 393us) server_negotiation.cc:299] Negotiation successful
0924 00:55:53.598464 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260924 00:55:53.598897 9515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:53.591838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:40955 (local address 127.8.64.130:36461)
0924 00:55:53.593332 (+ 1494us) negotiation.cc:107] Waiting for socket to connect
0924 00:55:53.593349 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0924 00:55:53.593474 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:55:53.593840 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:55:53.593843 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:55:53.593861 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:55:53.594235 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:53.594245 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:53.595304 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:53.595308 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:53.596756 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:53.596767 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:53.597929 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:53.597935 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:53.598146 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:53.598168 (+ 22us) client_negotiation.cc:769] Sending connection context
0924 00:55:53.598273 (+ 105us) client_negotiation.cc:240] Negotiation successful
0924 00:55:53.598435 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1367,"mutex_wait_us":46,"thread_start_us":1018,"threads_started":1}
I20260924 00:55:53.599311 9514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:53.591768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.131:37109 (local address 127.8.64.130:60573)
0924 00:55:53.593182 (+ 1414us) negotiation.cc:107] Waiting for socket to connect
0924 00:55:53.593198 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 00:55:53.593323 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:55:53.593649 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:55:53.593653 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:55:53.593685 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:55:53.594093 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:53.594105 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:53.595216 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:53.595225 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:53.596767 (+ 1542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:53.596778 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:53.597901 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:53.597911 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:53.598122 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:53.598156 (+ 34us) client_negotiation.cc:769] Sending connection context
0924 00:55:53.598230 (+ 74us) client_negotiation.cc:240] Negotiation successful
0924 00:55:53.598428 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1295,"mutex_wait_us":51,"thread_start_us":1073,"threads_started":1}
I20260924 00:55:53.599970 9172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05b58b9b730e4a449513d8e29256de0f" candidate_uuid: "7bc362f1de504681a8efec0e6417abc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c106e816b92e49e6bf085bf0f4758cbb" is_pre_election: true
I20260924 00:55:53.600235 9441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05b58b9b730e4a449513d8e29256de0f" candidate_uuid: "7bc362f1de504681a8efec0e6417abc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c95c6fcc1ff4406b81c0d791714860a" is_pre_election: true
I20260924 00:55:53.600684 9441 raft_consensus.cc:2454] T 05b58b9b730e4a449513d8e29256de0f P 4c95c6fcc1ff4406b81c0d791714860a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc362f1de504681a8efec0e6417abc9 in term 0.
I20260924 00:55:53.600888 9172 raft_consensus.cc:2454] T 05b58b9b730e4a449513d8e29256de0f P c106e816b92e49e6bf085bf0f4758cbb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc362f1de504681a8efec0e6417abc9 in term 0.
I20260924 00:55:53.601418 9241 leader_election.cc:302] T 05b58b9b730e4a449513d8e29256de0f P 7bc362f1de504681a8efec0e6417abc9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c95c6fcc1ff4406b81c0d791714860a, 7bc362f1de504681a8efec0e6417abc9; no voters:
I20260924 00:55:53.601814 9512 raft_consensus.cc:2781] T 05b58b9b730e4a449513d8e29256de0f P 7bc362f1de504681a8efec0e6417abc9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:55:53.601962 9512 raft_consensus.cc:482] T 05b58b9b730e4a449513d8e29256de0f P 7bc362f1de504681a8efec0e6417abc9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:55:53.602099 9512 raft_consensus.cc:3037] T 05b58b9b730e4a449513d8e29256de0f P 7bc362f1de504681a8efec0e6417abc9 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:55:53.604292 9512 raft_consensus.cc:504] T 05b58b9b730e4a449513d8e29256de0f P 7bc362f1de504681a8efec0e6417abc9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc362f1de504681a8efec0e6417abc9" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 33461 } } peers { permanent_uuid: "4c95c6fcc1ff4406b81c0d791714860a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 37109 } } peers { permanent_uuid: "c106e816b92e49e6bf085bf0f4758cbb" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 40955 } }
I20260924 00:55:53.604856 9512 leader_election.cc:288] T 05b58b9b730e4a449513d8e29256de0f P 7bc362f1de504681a8efec0e6417abc9 [CANDIDATE]: Term 1 election: Requested vote from peers 4c95c6fcc1ff4406b81c0d791714860a (127.8.64.131:37109), c106e816b92e49e6bf085bf0f4758cbb (127.8.64.129:40955)
I20260924 00:55:53.605356 9172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05b58b9b730e4a449513d8e29256de0f" candidate_uuid: "7bc362f1de504681a8efec0e6417abc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c106e816b92e49e6bf085bf0f4758cbb"
I20260924 00:55:53.605356 9441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05b58b9b730e4a449513d8e29256de0f" candidate_uuid: "7bc362f1de504681a8efec0e6417abc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c95c6fcc1ff4406b81c0d791714860a"
I20260924 00:55:53.605566 9441 raft_consensus.cc:3037] T 05b58b9b730e4a449513d8e29256de0f P 4c95c6fcc1ff4406b81c0d791714860a [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:55:53.605567 9172 raft_consensus.cc:3037] T 05b58b9b730e4a449513d8e29256de0f P c106e816b92e49e6bf085bf0f4758cbb [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:55:53.608758 9172 raft_consensus.cc:2454] T 05b58b9b730e4a449513d8e29256de0f P c106e816b92e49e6bf085bf0f4758cbb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc362f1de504681a8efec0e6417abc9 in term 1.
I20260924 00:55:53.608759 9441 raft_consensus.cc:2454] T 05b58b9b730e4a449513d8e29256de0f P 4c95c6fcc1ff4406b81c0d791714860a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc362f1de504681a8efec0e6417abc9 in term 1.
I20260924 00:55:53.609162 9241 leader_election.cc:302] T 05b58b9b730e4a449513d8e29256de0f P 7bc362f1de504681a8efec0e6417abc9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c95c6fcc1ff4406b81c0d791714860a, 7bc362f1de504681a8efec0e6417abc9; no voters:
I20260924 00:55:53.609457 9512 raft_consensus.cc:2781] T 05b58b9b730e4a449513d8e29256de0f P 7bc362f1de504681a8efec0e6417abc9 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:55:53.609962 9512 raft_consensus.cc:686] T 05b58b9b730e4a449513d8e29256de0f P 7bc362f1de504681a8efec0e6417abc9 [term 1 LEADER]: Becoming Leader. State: Replica: 7bc362f1de504681a8efec0e6417abc9, State: Running, Role: LEADER
I20260924 00:55:53.610476 9512 consensus_queue.cc:237] T 05b58b9b730e4a449513d8e29256de0f P 7bc362f1de504681a8efec0e6417abc9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc362f1de504681a8efec0e6417abc9" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 33461 } } peers { permanent_uuid: "4c95c6fcc1ff4406b81c0d791714860a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 37109 } } peers { permanent_uuid: "c106e816b92e49e6bf085bf0f4758cbb" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 40955 } }
I20260924 00:55:53.615183 9027 catalog_manager.cc:5495] T 05b58b9b730e4a449513d8e29256de0f P 7bc362f1de504681a8efec0e6417abc9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7bc362f1de504681a8efec0e6417abc9 (127.8.64.130). New cstate: current_term: 1 leader_uuid: "7bc362f1de504681a8efec0e6417abc9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc362f1de504681a8efec0e6417abc9" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 33461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c95c6fcc1ff4406b81c0d791714860a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 37109 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c106e816b92e49e6bf085bf0f4758cbb" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 40955 } health_report { overall_health: UNKNOWN } } }
Sep 24 00:55:53 dist-test-slave-mmhs krb5kdc[8982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.129@KRBTEST.COM
I20260924 00:55:53.691998 9503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:53.676180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38614 (local address 127.8.64.129:40955)
0924 00:55:53.676383 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:53.676390 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:53.676949 (+ 559us) server_negotiation.cc:407] Connection header received
0924 00:55:53.677075 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:53.677079 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:53.677160 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:53.677243 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:55:53.678713 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:53.679423 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:53.680819 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:53.681229 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:53.681320 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:53.685947 (+ 4627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:55:53.685961 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:55:53.685977 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:55:53.686029 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:55:53.689361 (+ 3332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:53.690102 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:53.690108 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:53.690110 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:53.690236 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:53.690639 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:53.690642 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:53.690644 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:53.690971 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:55:53.691178 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:53.691542 (+ 364us) server_negotiation.cc:299] Negotiation successful
0924 00:55:53.691687 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
Sep 24 00:55:53 dist-test-slave-mmhs krb5kdc[8982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211353, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 24 00:55:53 dist-test-slave-mmhs krb5kdc[8982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211353, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.64.129@KRBTEST.COM
I20260924 00:55:53.738560 9503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:53.724206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38624 (local address 127.8.64.129:40955)
0924 00:55:53.724417 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:53.724426 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:53.724768 (+ 342us) server_negotiation.cc:407] Connection header received
0924 00:55:53.725010 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:53.725016 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:53.725096 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:53.725202 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:55:53.726835 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:53.727558 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:53.728856 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:53.729287 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:53.729455 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:53.733783 (+ 4328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:55:53.733787 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:55:53.733791 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:55:53.733823 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:55:53.736327 (+ 2504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:53.736891 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:53.736895 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:53.736897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:53.737016 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:53.737390 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:53.737394 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:53.737397 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:53.737667 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:55:53.737894 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:53.738077 (+ 183us) server_negotiation.cc:299] Negotiation successful
0924 00:55:53.738231 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
Sep 24 00:55:53 dist-test-slave-mmhs krb5kdc[8982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 24 00:55:53 dist-test-slave-mmhs krb5kdc[8982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.129@KRBTEST.COM
I20260924 00:55:53.781752 9503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:53.768235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38632 (local address 127.8.64.129:40955)
0924 00:55:53.768592 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:53.768600 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:53.768732 (+ 132us) server_negotiation.cc:407] Connection header received
0924 00:55:53.768961 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:53.768965 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:53.769033 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:53.769146 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:55:53.770564 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:53.771220 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:53.772386 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:53.772800 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:53.772893 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:53.776618 (+ 3725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:55:53.776623 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:55:53.776626 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:55:53.776655 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:55:53.778997 (+ 2342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:53.779730 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:53.779734 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:53.779736 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:53.779847 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:53.780147 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:53.780150 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:53.780151 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:53.780415 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:55:53.780612 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:53.780805 (+ 193us) server_negotiation.cc:299] Negotiation successful
0924 00:55:53.780966 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
W20260924 00:55:53.783589 9132 tablet_service.cc:2908] Rejecting scan request for tablet 05b58b9b730e4a449513d8e29256de0f: Uninitialized: clean time has not yet been initialized
I20260924 00:55:53.787284 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 9089
I20260924 00:55:53.799214 9503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:53.787973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38644 (local address 127.8.64.129:40955)
0924 00:55:53.788156 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:53.788160 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:53.788481 (+ 321us) server_negotiation.cc:407] Connection header received
0924 00:55:53.788620 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:53.788624 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:53.788688 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:53.788766 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:55:53.789802 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:53.790511 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:53.791669 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:53.792057 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:53.792155 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:53.794106 (+ 1951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:55:53.794111 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:55:53.794123 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:55:53.794150 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:55:53.796700 (+ 2550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:53.797327 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:53.797332 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:53.797335 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:53.797456 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:53.797819 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:53.797825 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:53.797828 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:53.798103 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:55:53.798410 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:53.798693 (+ 283us) server_negotiation.cc:299] Negotiation successful
0924 00:55:53.798893 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260924 00:55:53.799788 9213 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:55:53.890075 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 9089
I20260924 00:55:53.907280 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 9224
Sep 24 00:55:53 dist-test-slave-mmhs krb5kdc[8982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.130@KRBTEST.COM
I20260924 00:55:53.921419 9530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:53.908044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49356 (local address 127.8.64.130:33461)
0924 00:55:53.908507 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:53.908512 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:53.908528 (+ 16us) server_negotiation.cc:407] Connection header received
0924 00:55:53.908589 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:53.908593 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:53.908650 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:53.908743 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:55:53.909868 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:53.910667 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:53.911929 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:53.912333 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:53.912421 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:53.916222 (+ 3801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:55:53.916236 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:55:53.916252 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:55:53.916293 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:55:53.919232 (+ 2939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:53.919801 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:53.919808 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:53.919810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:53.919926 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:53.920234 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:53.920237 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:53.920238 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:53.920567 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:55:53.920775 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:53.921013 (+ 238us) server_negotiation.cc:299] Negotiation successful
0924 00:55:53.921171 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":262,"threads_started":1}
I20260924 00:55:53.922436 9348 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:55:54.012116 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 9224
I20260924 00:55:54.029362 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 9358
Sep 24 00:55:54 dist-test-slave-mmhs krb5kdc[8982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.131@KRBTEST.COM
I20260924 00:55:54.044982 9502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:54.030059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59218 (local address 127.8.64.131:37109)
0924 00:55:54.030234 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:54.030242 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:54.030282 (+ 40us) server_negotiation.cc:407] Connection header received
0924 00:55:54.030533 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:54.030539 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:54.030614 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:54.030722 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:55:54.031938 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:54.032710 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:54.033957 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:54.034423 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:54.034555 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:54.039217 (+ 4662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:55:54.039233 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:55:54.039250 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:55:54.039294 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:55:54.042011 (+ 2717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:54.042941 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:54.042950 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:54.042954 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:54.043102 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:54.043474 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:54.043479 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:54.043482 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:54.043822 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:55:54.044113 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:54.044422 (+ 309us) server_negotiation.cc:299] Negotiation successful
0924 00:55:54.044632 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260924 00:55:54.045668 9481 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:55:54.130470 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 9358
I20260924 00:55:54.146729 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 8995
I20260924 00:55:54.147993 9058 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:55:54.253077 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 8995
2026-09-24T00:55:54Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4274 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:55:54 dist-test-slave-mmhs krb5kdc[9536](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:55:54 dist-test-slave-mmhs krb5kdc[9536](info): set up 2 sockets
Sep 24 00:55:54 dist-test-slave-mmhs krb5kdc[9536](info): commencing operation
krb5kdc: starting...
W20260924 00:55:55.352082 8450 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.000s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Sep 24 00:55:55 dist-test-slave-mmhs krb5kdc[9536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:55:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:55:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.64.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
I20260924 00:55:55.532265 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:39865
--webserver_interface=127.8.64.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:33955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:39865
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:55:55.847023 9549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:55:55.847354 9549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:55:55.847541 9549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:55:55.854684 9549 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:55:55.854818 9549 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260924 00:55:55.854913 9549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:55:55.855041 9549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:55:55.855125 9549 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:55:55.855214 9549 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:55:55.864883 9549 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:33955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:39865
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:39865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:55:55.866096 9549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:55:55.867378 9549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:55:55.878545 9554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:55:55.878593 9557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:55:55.878619 9555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:55:55.879654 9549 server_base.cc:1034] running on GCE node
I20260924 00:55:55.880425 9549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:55:55.881819 9549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:55:55.883080 9549 hybrid_clock.cc:648] HybridClock initialized: now 1790211355883010 us; error 42 us; skew 500 ppm
I20260924 00:55:55.883417 9549 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:55:55 dist-test-slave-mmhs krb5kdc[9536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211355, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:55:55.886458 9549 init.cc:377] Logged in from keytab as kudu/127.8.64.190@KRBTEST.COM (short username kudu)
I20260924 00:55:55.887900 9549 webserver.cc:460] Webserver started at http://127.8.64.190:38645/ using document root <none> and password file <none>
I20260924 00:55:55.888622 9549 fs_manager.cc:359] Metadata directory not provided
I20260924 00:55:55.888756 9549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:55:55.889081 9549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:55:55.892112 9549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "c3573e3a5c4e473a927e9723abed68de"
format_stamp: "Formatted at 2026-09-24 00:55:55 on dist-test-slave-mmhs"
server_key: "c2d655161267d30d2e17d636808b999a"
server_key_iv: "0669396b4cc17c944b20f0da4cf0c138"
server_key_version: "encryptionkey@0"
I20260924 00:55:55.892938 9549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "c3573e3a5c4e473a927e9723abed68de"
format_stamp: "Formatted at 2026-09-24 00:55:55 on dist-test-slave-mmhs"
server_key: "c2d655161267d30d2e17d636808b999a"
server_key_iv: "0669396b4cc17c944b20f0da4cf0c138"
server_key_version: "encryptionkey@0"
I20260924 00:55:55.898258 9549 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260924 00:55:55.901834 9564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:55:55.903330 9549 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 00:55:55.903523 9549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "c3573e3a5c4e473a927e9723abed68de"
format_stamp: "Formatted at 2026-09-24 00:55:55 on dist-test-slave-mmhs"
server_key: "c2d655161267d30d2e17d636808b999a"
server_key_iv: "0669396b4cc17c944b20f0da4cf0c138"
server_key_version: "encryptionkey@0"
I20260924 00:55:55.903755 9549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:55:55.921583 9549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:55:55.925719 9549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:55:55.926012 9549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:55:55.948045 9549 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.190:39865
I20260924 00:55:55.948083 9626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.190:39865 every 8 connection(s)
I20260924 00:55:55.949400 9549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 00:55:55.953081 9627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:55:55.959417 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 9549
I20260924 00:55:55.959645 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 00:55:55.960278 8450 external_mini_cluster.cc:1428] Setting key e8fc7f3c384df927043dfc1caaa1b3b0
I20260924 00:55:55.963696 9627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3573e3a5c4e473a927e9723abed68de: Bootstrap starting.
I20260924 00:55:55.967612 9627 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3573e3a5c4e473a927e9723abed68de: Neither blocks nor log segments found. Creating new log.
I20260924 00:55:55.968680 9627 log.cc:824] T 00000000000000000000000000000000 P c3573e3a5c4e473a927e9723abed68de: Log is configured to *not* fsync() on all Append() calls
I20260924 00:55:55.971379 9627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3573e3a5c4e473a927e9723abed68de: No bootstrap required, opened a new log
Sep 24 00:55:55 dist-test-slave-mmhs krb5kdc[9536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:55:55.979398 9627 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3573e3a5c4e473a927e9723abed68de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3573e3a5c4e473a927e9723abed68de" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 39865 } }
I20260924 00:55:55.979763 9627 raft_consensus.cc:374] T 00000000000000000000000000000000 P c3573e3a5c4e473a927e9723abed68de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:55:55.979892 9627 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3573e3a5c4e473a927e9723abed68de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3573e3a5c4e473a927e9723abed68de, State: Initialized, Role: FOLLOWER
I20260924 00:55:55.980494 9627 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3573e3a5c4e473a927e9723abed68de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3573e3a5c4e473a927e9723abed68de" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 39865 } }
I20260924 00:55:55.980758 9627 raft_consensus.cc:388] T 00000000000000000000000000000000 P c3573e3a5c4e473a927e9723abed68de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:55:55.980906 9627 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3573e3a5c4e473a927e9723abed68de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:55:55.981110 9627 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3573e3a5c4e473a927e9723abed68de [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:55:55.982779 9630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:55.962235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43656 (local address 127.8.64.190:39865)
0924 00:55:55.963129 (+ 894us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:55.963145 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:55.963201 (+ 56us) server_negotiation.cc:407] Connection header received
0924 00:55:55.965633 (+ 2432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:55.965664 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:55.966179 (+ 515us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:55.966615 (+ 436us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:55:55.967927 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:55.969255 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:55.970534 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:55.971179 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:55.971380 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:55.975385 (+ 4005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:55:55.975392 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:55:55.975406 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:55:55.975450 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:55:55.978580 (+ 3130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:55.979242 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:55.979248 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:55.979255 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:55.979406 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:55.979821 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:55.979843 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:55.979849 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:55.980566 (+ 717us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:55:55.980863 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:55.981271 (+ 408us) server_negotiation.cc:299] Negotiation successful
0924 00:55:55.981687 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":356,"threads_started":1}
I20260924 00:55:55.983654 9627 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3573e3a5c4e473a927e9723abed68de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3573e3a5c4e473a927e9723abed68de" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 39865 } }
I20260924 00:55:55.984192 9627 leader_election.cc:302] T 00000000000000000000000000000000 P c3573e3a5c4e473a927e9723abed68de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c3573e3a5c4e473a927e9723abed68de; no voters:
I20260924 00:55:55.984787 9627 leader_election.cc:288] T 00000000000000000000000000000000 P c3573e3a5c4e473a927e9723abed68de [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:55:55.984969 9632 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3573e3a5c4e473a927e9723abed68de [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:55:55.986161 9632 raft_consensus.cc:686] T 00000000000000000000000000000000 P c3573e3a5c4e473a927e9723abed68de [term 1 LEADER]: Becoming Leader. State: Replica: c3573e3a5c4e473a927e9723abed68de, State: Running, Role: LEADER
I20260924 00:55:55.986656 9632 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3573e3a5c4e473a927e9723abed68de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3573e3a5c4e473a927e9723abed68de" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 39865 } }
I20260924 00:55:55.987059 9627 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3573e3a5c4e473a927e9723abed68de [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:55:55.990396 9633 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3573e3a5c4e473a927e9723abed68de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c3573e3a5c4e473a927e9723abed68de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3573e3a5c4e473a927e9723abed68de" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 39865 } } }
I20260924 00:55:55.990371 9634 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3573e3a5c4e473a927e9723abed68de [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3573e3a5c4e473a927e9723abed68de. Latest consensus state: current_term: 1 leader_uuid: "c3573e3a5c4e473a927e9723abed68de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3573e3a5c4e473a927e9723abed68de" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 39865 } } }
I20260924 00:55:55.990859 9633 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3573e3a5c4e473a927e9723abed68de [sys.catalog]: This master's current role is: LEADER
I20260924 00:55:55.990886 9634 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3573e3a5c4e473a927e9723abed68de [sys.catalog]: This master's current role is: LEADER
I20260924 00:55:55.991708 9639 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:55:55.997200 9639 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:55:56.007175 9639 catalog_manager.cc:1347] Generated new cluster ID: e3c338e5d13a406fb6b144d643d20c2e
I20260924 00:55:56.007330 9639 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:55:56.021879 9639 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:55:56.023268 9639 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:55:56.032310 9639 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c3573e3a5c4e473a927e9723abed68de: Generated new TSK 0
I20260924 00:55:56.033375 9639 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.64.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab.
Entry for principal kudu/127.8.64.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab.
I20260924 00:55:56.090318 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.129:0
--local_ip_for_outbound_sockets=127.8.64.129
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:39865
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab
--principal=kudu/127.8.64.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:33955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:55:56.413842 9653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:55:56.414141 9653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:55:56.414364 9653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:55:56.422488 9653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:55:56.422643 9653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:55:56.422786 9653 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 00:55:56.422925 9653 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.129
I20260924 00:55:56.432922 9653 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:33955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab
--principal=kudu/127.8.64.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.64.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.64.190:39865
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.129
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:55:56.434273 9653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:55:56.435729 9653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:55:56.449317 9659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:55:56.449445 9661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:55:56.449353 9658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:55:56.450362 9653 server_base.cc:1034] running on GCE node
I20260924 00:55:56.450979 9653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:55:56.451956 9653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:55:56.453182 9653 hybrid_clock.cc:648] HybridClock initialized: now 1790211356453114 us; error 42 us; skew 500 ppm
I20260924 00:55:56.453497 9653 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:55:56 dist-test-slave-mmhs krb5kdc[9536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211356, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:55:56.456733 9653 init.cc:377] Logged in from keytab as kudu/127.8.64.129@KRBTEST.COM (short username kudu)
I20260924 00:55:56.458220 9653 webserver.cc:460] Webserver started at http://127.8.64.129:39883/ using document root <none> and password file <none>
I20260924 00:55:56.458935 9653 fs_manager.cc:359] Metadata directory not provided
I20260924 00:55:56.459069 9653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:55:56.459381 9653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:55:56.462177 9653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-0/data/instance:
uuid: "d06b687eed2a433781313535a31c682a"
format_stamp: "Formatted at 2026-09-24 00:55:56 on dist-test-slave-mmhs"
server_key: "9ed35166bd7e2270b4db0639aa5ab6dc"
server_key_iv: "629dafb725e81dd3cc517fd693634081"
server_key_version: "encryptionkey@0"
I20260924 00:55:56.463035 9653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance:
uuid: "d06b687eed2a433781313535a31c682a"
format_stamp: "Formatted at 2026-09-24 00:55:56 on dist-test-slave-mmhs"
server_key: "9ed35166bd7e2270b4db0639aa5ab6dc"
server_key_iv: "629dafb725e81dd3cc517fd693634081"
server_key_version: "encryptionkey@0"
I20260924 00:55:56.468045 9653 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260924 00:55:56.471405 9668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:55:56.472693 9653 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260924 00:55:56.472877 9653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-0/wal
uuid: "d06b687eed2a433781313535a31c682a"
format_stamp: "Formatted at 2026-09-24 00:55:56 on dist-test-slave-mmhs"
server_key: "9ed35166bd7e2270b4db0639aa5ab6dc"
server_key_iv: "629dafb725e81dd3cc517fd693634081"
server_key_version: "encryptionkey@0"
I20260924 00:55:56.473098 9653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:55:56.486195 9653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:55:56.490743 9653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:55:56.491019 9653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:55:56.493620 9653 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:55:56.493731 9653 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:55:56.493916 9653 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:55:56.494082 9653 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 00:55:56 dist-test-slave-mmhs krb5kdc[9536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211356, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:55:56.522364 9678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:56.494884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:39865 (local address 127.8.64.129:51511)
0924 00:55:56.496023 (+ 1139us) negotiation.cc:107] Waiting for socket to connect
0924 00:55:56.496074 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0924 00:55:56.498954 (+ 2880us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:55:56.499914 (+ 960us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:55:56.499927 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:55:56.500934 (+ 1007us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:55:56.502261 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:56.502325 (+ 64us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:56.503651 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:56.503663 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:56.506583 (+ 2920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:56.506619 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:56.507547 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:56.507566 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:56.507970 (+ 404us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:56.508931 (+ 961us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:55:56.508973 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:55:56.512800 (+ 3827us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:55:56.516940 (+ 4140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:55:56.516949 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:55:56.516966 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:55:56.517643 (+ 677us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:55:56.518047 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:55:56.518052 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:55:56.518056 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:55:56.518255 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:55:56.519099 (+ 844us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:55:56.519104 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:55:56.519472 (+ 368us) client_negotiation.cc:769] Sending connection context
0924 00:55:56.519872 (+ 400us) client_negotiation.cc:240] Negotiation successful
0924 00:55:56.520514 (+ 642us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":743,"thread_start_us":436,"threads_started":1}
I20260924 00:55:56.523020 9677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:56.494144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:51511 (local address 127.8.64.190:39865)
0924 00:55:56.496854 (+ 2710us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:56.496858 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:56.496880 (+ 22us) server_negotiation.cc:407] Connection header received
0924 00:55:56.499349 (+ 2469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:56.499355 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:56.499447 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:56.499549 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:55:56.502549 (+ 3000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:56.503459 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:56.506788 (+ 3329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:56.507251 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:56.507354 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:56.513095 (+ 5741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:55:56.513100 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:55:56.513104 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:55:56.513135 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:55:56.516798 (+ 3663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:56.517794 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:56.517799 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:56.517803 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:56.517930 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:56.518431 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:56.518440 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:56.518446 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:56.518931 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:55:56.519135 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:56.522440 (+ 3305us) server_negotiation.cc:299] Negotiation successful
0924 00:55:56.522676 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2596,"thread_start_us":2487,"threads_started":1}
I20260924 00:55:56.545475 9677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:56.532659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:56949 (local address 127.8.64.190:39865)
0924 00:55:56.532812 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:56.532816 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:56.533739 (+ 923us) server_negotiation.cc:407] Connection header received
0924 00:55:56.534017 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:56.534026 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:56.534085 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:56.534187 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:55:56.535416 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:56.536143 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:56.537436 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:56.537842 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:56.537981 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:56.540208 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:55:56.540213 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:55:56.540216 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:55:56.540245 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:55:56.542869 (+ 2624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:56.543555 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:56.543561 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:56.543567 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:56.543695 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:56.544061 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:56.544066 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:56.544069 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:56.544328 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:55:56.544510 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:56.544889 (+ 379us) server_negotiation.cc:299] Negotiation successful
0924 00:55:56.545118 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260924 00:55:56.545581 9745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:56.532719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:39865 (local address 127.8.64.129:56949)
0924 00:55:56.533616 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0924 00:55:56.533642 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0924 00:55:56.533864 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:55:56.534214 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:55:56.534218 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:55:56.534969 (+ 751us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:55:56.535275 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:56.535285 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:56.536293 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:56.536302 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:56.537299 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:56.537310 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:56.538024 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:56.538031 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:56.538195 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:56.538890 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:55:56.538921 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:55:56.540012 (+ 1091us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:55:56.543007 (+ 2995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:55:56.543014 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:55:56.543020 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:55:56.543412 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:55:56.543814 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:55:56.543819 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:55:56.543823 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:55:56.543942 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:55:56.544456 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:55:56.544462 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:55:56.544654 (+ 192us) client_negotiation.cc:769] Sending connection context
0924 00:55:56.544805 (+ 151us) client_negotiation.cc:240] Negotiation successful
0924 00:55:56.545118 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":688,"thread_start_us":503,"threads_started":1}
I20260924 00:55:56.557816 9653 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.129:43265
I20260924 00:55:56.557823 9789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.129:43265 every 8 connection(s)
I20260924 00:55:56.559100 9653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
I20260924 00:55:56.559690 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 9653
I20260924 00:55:56.559933 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance
I20260924 00:55:56.560745 8450 external_mini_cluster.cc:1428] Setting key b4f97b4c9754085a9ef12c1380709cf6
I20260924 00:55:56.561486 9790 heartbeater.cc:347] Connected to a master server at 127.8.64.190:39865
I20260924 00:55:56.561718 9790 heartbeater.cc:464] Registering TS with master...
I20260924 00:55:56.562460 9790 heartbeater.cc:511] Master 127.8.64.190:39865 requested a full tablet report, sending...
I20260924 00:55:56.564549 9581 ts_manager.cc:194] Registered new tserver with Master: d06b687eed2a433781313535a31c682a (127.8.64.129:43265)
I20260924 00:55:56.566037 9581 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.129@KRBTEST.COM'} at 127.8.64.129:51511
WARNING: no policy specified for kudu/127.8.64.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab.
Entry for principal kudu/127.8.64.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab.
I20260924 00:55:56.602623 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.130:0
--local_ip_for_outbound_sockets=127.8.64.130
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:39865
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab
--principal=kudu/127.8.64.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:33955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:55:56.923275 9794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:55:56.923540 9794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:55:56.923635 9794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:55:56.931190 9794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:55:56.931298 9794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:55:56.931380 9794 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 00:55:56.931458 9794 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.130
I20260924 00:55:56.940878 9794 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:33955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab
--principal=kudu/127.8.64.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.64.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.64.190:39865
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.130
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:55:56.942051 9794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:55:56.943308 9794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:55:56.956677 9799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:55:56.956655 9800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:55:56.957262 9802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:55:56.957691 9794 server_base.cc:1034] running on GCE node
I20260924 00:55:56.958390 9794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:55:56.959503 9794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:55:56.960763 9794 hybrid_clock.cc:648] HybridClock initialized: now 1790211356960666 us; error 58 us; skew 500 ppm
I20260924 00:55:56.961120 9794 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:55:56 dist-test-slave-mmhs krb5kdc[9536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211356, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:55:56.964687 9794 init.cc:377] Logged in from keytab as kudu/127.8.64.130@KRBTEST.COM (short username kudu)
I20260924 00:55:56.966176 9794 webserver.cc:460] Webserver started at http://127.8.64.130:38189/ using document root <none> and password file <none>
I20260924 00:55:56.966890 9794 fs_manager.cc:359] Metadata directory not provided
I20260924 00:55:56.967018 9794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:55:56.967362 9794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:55:56.970443 9794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-1/data/instance:
uuid: "e73cb1b3c0c74cecafa9b41b52628f77"
format_stamp: "Formatted at 2026-09-24 00:55:56 on dist-test-slave-mmhs"
server_key: "dfe2f3a0b6d35de58050047957e04607"
server_key_iv: "464aac02667ef5782db5052d2fa06106"
server_key_version: "encryptionkey@0"
I20260924 00:55:56.971248 9794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance:
uuid: "e73cb1b3c0c74cecafa9b41b52628f77"
format_stamp: "Formatted at 2026-09-24 00:55:56 on dist-test-slave-mmhs"
server_key: "dfe2f3a0b6d35de58050047957e04607"
server_key_iv: "464aac02667ef5782db5052d2fa06106"
server_key_version: "encryptionkey@0"
I20260924 00:55:56.976171 9794 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260924 00:55:56.979826 9809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:55:56.981055 9794 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260924 00:55:56.981208 9794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-1/wal
uuid: "e73cb1b3c0c74cecafa9b41b52628f77"
format_stamp: "Formatted at 2026-09-24 00:55:56 on dist-test-slave-mmhs"
server_key: "dfe2f3a0b6d35de58050047957e04607"
server_key_iv: "464aac02667ef5782db5052d2fa06106"
server_key_version: "encryptionkey@0"
I20260924 00:55:56.981420 9794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:55:57.010103 9794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:55:57.014662 9794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:55:57.014946 9794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:55:57.017526 9794 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:55:57.017637 9794 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:55:57.017807 9794 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:55:57.017889 9794 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 00:55:57 dist-test-slave-mmhs krb5kdc[9536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211356, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.130@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:55:57.043154 9677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:57.018236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:51373 (local address 127.8.64.190:39865)
0924 00:55:57.018478 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:57.018488 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:57.019983 (+ 1495us) server_negotiation.cc:407] Connection header received
0924 00:55:57.022369 (+ 2386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:57.022376 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:57.022464 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:57.022542 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:55:57.025270 (+ 2728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:57.026622 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:57.029305 (+ 2683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:57.029962 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:57.030101 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:57.035595 (+ 5494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:55:57.035606 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:55:57.035614 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:55:57.035671 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:55:57.039675 (+ 4004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:57.040464 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:57.040472 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:57.040478 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:57.040679 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:57.041101 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:57.041108 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:57.041114 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:57.041571 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:55:57.041747 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:57.042470 (+ 723us) server_negotiation.cc:299] Negotiation successful
0924 00:55:57.042712 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260924 00:55:57.044369 9818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:57.018920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:39865 (local address 127.8.64.130:51373)
0924 00:55:57.019807 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0924 00:55:57.019855 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0924 00:55:57.021934 (+ 2079us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:55:57.023086 (+ 1152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:55:57.023094 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:55:57.023991 (+ 897us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:55:57.024967 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:57.024997 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:57.026844 (+ 1847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:57.026850 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:57.029065 (+ 2215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:57.029093 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:57.030269 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:57.030280 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:57.030656 (+ 376us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:57.031492 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:55:57.031538 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:55:57.035228 (+ 3690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:55:57.039850 (+ 4622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:55:57.039861 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:55:57.039885 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:55:57.040332 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:55:57.040806 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:55:57.040809 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:55:57.040811 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:55:57.040996 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:55:57.041707 (+ 711us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:55:57.041714 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:55:57.042038 (+ 324us) client_negotiation.cc:769] Sending connection context
0924 00:55:57.042271 (+ 233us) client_negotiation.cc:240] Negotiation successful
0924 00:55:57.043007 (+ 736us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":383,"threads_started":1}
I20260924 00:55:57.062676 9794 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.130:46619
I20260924 00:55:57.062680 9929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.130:46619 every 8 connection(s)
I20260924 00:55:57.063956 9794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
I20260924 00:55:57.066461 9930 heartbeater.cc:347] Connected to a master server at 127.8.64.190:39865
I20260924 00:55:57.066803 9930 heartbeater.cc:464] Registering TS with master...
I20260924 00:55:57.066990 9905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:57.051966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:39865 (local address 127.8.64.130:59141)
0924 00:55:57.052769 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0924 00:55:57.052788 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0924 00:55:57.053024 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:55:57.053485 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:55:57.053489 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:55:57.053930 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:55:57.054615 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:57.054626 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:57.055882 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:57.055890 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:57.057611 (+ 1721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:57.057627 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:57.058427 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:57.058433 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:57.058628 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:57.059449 (+ 821us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:55:57.059485 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:55:57.060874 (+ 1389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:55:57.064191 (+ 3317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:55:57.064197 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:55:57.064201 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:55:57.064610 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:55:57.064997 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:55:57.065002 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:55:57.065005 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:55:57.065146 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:55:57.065716 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:55:57.065720 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:55:57.065898 (+ 178us) client_negotiation.cc:769] Sending connection context
0924 00:55:57.066042 (+ 144us) client_negotiation.cc:240] Negotiation successful
0924 00:55:57.066463 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":406,"threads_started":1}
I20260924 00:55:57.067209 9677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:57.051919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:59141 (local address 127.8.64.190:39865)
0924 00:55:57.052118 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:57.052126 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:57.052892 (+ 766us) server_negotiation.cc:407] Connection header received
0924 00:55:57.053200 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:57.053209 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:57.053318 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:57.053450 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:55:57.054766 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:57.055679 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:57.057762 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:57.058199 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:57.058361 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:57.061059 (+ 2698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:55:57.061066 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:55:57.061070 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:55:57.061100 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:55:57.064045 (+ 2945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:57.064743 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:57.064751 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:57.064756 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:57.064884 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:57.065262 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:57.065270 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:57.065274 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:57.065595 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:55:57.065747 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:57.066653 (+ 906us) server_negotiation.cc:299] Negotiation successful
0924 00:55:57.066876 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260924 00:55:57.067865 9930 heartbeater.cc:511] Master 127.8.64.190:39865 requested a full tablet report, sending...
I20260924 00:55:57.069504 9581 ts_manager.cc:194] Registered new tserver with Master: e73cb1b3c0c74cecafa9b41b52628f77 (127.8.64.130:46619)
I20260924 00:55:57.070669 9581 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.130@KRBTEST.COM'} at 127.8.64.130:51373
I20260924 00:55:57.070735 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 9794
I20260924 00:55:57.070886 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance
I20260924 00:55:57.071439 8450 external_mini_cluster.cc:1428] Setting key f5c8d98a9cf977cfaa7a2e537dca6c2d
WARNING: no policy specified for kudu/127.8.64.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab.
Entry for principal kudu/127.8.64.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab.
I20260924 00:55:57.115253 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.131:0
--local_ip_for_outbound_sockets=127.8.64.131
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:39865
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab
--principal=kudu/127.8.64.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:33955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:55:57.437919 9934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:55:57.438236 9934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:55:57.438424 9934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:55:57.445456 9934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:55:57.445604 9934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:55:57.445752 9934 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 00:55:57.445894 9934 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.131
I20260924 00:55:57.455286 9934 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:33955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab
--principal=kudu/127.8.64.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.64.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.64.190:39865
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.131
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:55:57.456557 9934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:55:57.457887 9934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:55:57.470928 9940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:55:57.471022 9942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:55:57.471149 9939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:55:57.472618 9934 server_base.cc:1034] running on GCE node
I20260924 00:55:57.473317 9934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:55:57.474265 9934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:55:57.475592 9934 hybrid_clock.cc:648] HybridClock initialized: now 1790211357475514 us; error 47 us; skew 500 ppm
I20260924 00:55:57.475950 9934 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:55:57 dist-test-slave-mmhs krb5kdc[9536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211357, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:55:57.479405 9934 init.cc:377] Logged in from keytab as kudu/127.8.64.131@KRBTEST.COM (short username kudu)
I20260924 00:55:57.480875 9934 webserver.cc:460] Webserver started at http://127.8.64.131:36991/ using document root <none> and password file <none>
I20260924 00:55:57.481595 9934 fs_manager.cc:359] Metadata directory not provided
I20260924 00:55:57.481727 9934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:55:57.482044 9934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:55:57.485082 9934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-2/data/instance:
uuid: "e4f058a495e44e7cbcc73bee4d367c7d"
format_stamp: "Formatted at 2026-09-24 00:55:57 on dist-test-slave-mmhs"
server_key: "c3c038d274419390104327981e2debe4"
server_key_iv: "344dfed70585237da1a4d7a7cec34a43"
server_key_version: "encryptionkey@0"
I20260924 00:55:57.485996 9934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance:
uuid: "e4f058a495e44e7cbcc73bee4d367c7d"
format_stamp: "Formatted at 2026-09-24 00:55:57 on dist-test-slave-mmhs"
server_key: "c3c038d274419390104327981e2debe4"
server_key_iv: "344dfed70585237da1a4d7a7cec34a43"
server_key_version: "encryptionkey@0"
I20260924 00:55:57.491446 9934 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260924 00:55:57.494896 9949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:55:57.496397 9934 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260924 00:55:57.496574 9934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-2/wal
uuid: "e4f058a495e44e7cbcc73bee4d367c7d"
format_stamp: "Formatted at 2026-09-24 00:55:57 on dist-test-slave-mmhs"
server_key: "c3c038d274419390104327981e2debe4"
server_key_iv: "344dfed70585237da1a4d7a7cec34a43"
server_key_version: "encryptionkey@0"
I20260924 00:55:57.496794 9934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:55:57.511196 9934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:55:57.515453 9934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:55:57.515758 9934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:55:57.518378 9934 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:55:57.518488 9934 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:55:57.518713 9934 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:55:57.518797 9934 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 00:55:57 dist-test-slave-mmhs krb5kdc[9536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211357, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.131@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:55:57.543903 9677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:57.518797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:41007 (local address 127.8.64.190:39865)
0924 00:55:57.518983 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:57.518993 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:57.520478 (+ 1485us) server_negotiation.cc:407] Connection header received
0924 00:55:57.523505 (+ 3027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:57.523513 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:57.523613 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:57.523737 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:55:57.526373 (+ 2636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:57.527386 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:57.530676 (+ 3290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:57.531062 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:57.531164 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:57.536525 (+ 5361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:55:57.536534 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:55:57.536541 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:55:57.536604 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:55:57.539751 (+ 3147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:57.540896 (+ 1145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:57.540905 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:57.540911 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:57.541099 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:57.541651 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:57.541656 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:57.541660 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:57.542052 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:55:57.542236 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:57.543326 (+ 1090us) server_negotiation.cc:299] Negotiation successful
0924 00:55:57.543546 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260924 00:55:57.545346 9958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:57.519449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:39865 (local address 127.8.64.131:41007)
0924 00:55:57.520283 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0924 00:55:57.520347 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0924 00:55:57.523178 (+ 2831us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:55:57.524045 (+ 867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:55:57.524061 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:55:57.525096 (+ 1035us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:55:57.526139 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:57.526164 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:57.527554 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:57.527562 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:57.530492 (+ 2930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:57.530512 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:57.531319 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:57.531328 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:57.531681 (+ 353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:57.532486 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:55:57.532524 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:55:57.536232 (+ 3708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:55:57.539904 (+ 3672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:55:57.539919 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:55:57.539943 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:55:57.540711 (+ 768us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:55:57.541245 (+ 534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:55:57.541255 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:55:57.541260 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:55:57.541514 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:55:57.542202 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:55:57.542210 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:55:57.542814 (+ 604us) client_negotiation.cc:769] Sending connection context
0924 00:55:57.543210 (+ 396us) client_negotiation.cc:240] Negotiation successful
0924 00:55:57.543754 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":333,"threads_started":1}
I20260924 00:55:57.568439 9677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:57.551615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:33861 (local address 127.8.64.190:39865)
0924 00:55:57.551846 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:57.551852 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:57.552302 (+ 450us) server_negotiation.cc:407] Connection header received
0924 00:55:57.552499 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:57.552502 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:57.552575 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:57.552655 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:55:57.554106 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:57.554911 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:57.557459 (+ 2548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:57.558053 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:57.558191 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:57.560707 (+ 2516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:55:57.560717 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:55:57.560725 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:55:57.560774 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:55:57.565083 (+ 4309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:57.566118 (+ 1035us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:57.566126 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:57.566133 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:57.566349 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:57.566815 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:57.566822 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:57.566826 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:57.567182 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:55:57.567332 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:57.567848 (+ 516us) server_negotiation.cc:299] Negotiation successful
0924 00:55:57.568078 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260924 00:55:57.568550 10023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:57.551234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:39865 (local address 127.8.64.131:33861)
0924 00:55:57.552180 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0924 00:55:57.552205 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0924 00:55:57.552375 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:55:57.552723 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:55:57.552730 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:55:57.553462 (+ 732us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:55:57.553921 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:57.553937 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:57.555063 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:57.555072 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:57.557215 (+ 2143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:57.557235 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:57.558475 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:57.558484 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:57.558674 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:57.559407 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:55:57.559427 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:55:57.560456 (+ 1029us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:55:57.565242 (+ 4786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:55:57.565251 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:55:57.565256 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:55:57.565970 (+ 714us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:55:57.566489 (+ 519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:55:57.566496 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:55:57.566501 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:55:57.566681 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:55:57.567416 (+ 735us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:55:57.567422 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:55:57.567597 (+ 175us) client_negotiation.cc:769] Sending connection context
0924 00:55:57.567751 (+ 154us) client_negotiation.cc:240] Negotiation successful
0924 00:55:57.568078 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":730,"thread_start_us":572,"threads_started":1}
I20260924 00:55:57.568913 9790 heartbeater.cc:503] Master 127.8.64.190:39865 was elected leader, sending a full tablet report...
I20260924 00:55:57.582129 9934 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.131:40187
I20260924 00:55:57.582147 10069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.131:40187 every 8 connection(s)
I20260924 00:55:57.583384 9934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
I20260924 00:55:57.585327 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 9934
I20260924 00:55:57.585455 10070 heartbeater.cc:347] Connected to a master server at 127.8.64.190:39865
I20260924 00:55:57.585537 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance
I20260924 00:55:57.585732 10070 heartbeater.cc:464] Registering TS with master...
I20260924 00:55:57.586151 8450 external_mini_cluster.cc:1428] Setting key e9ea12f85e6bb9ba3a690db23407c1ce
I20260924 00:55:57.586589 10070 heartbeater.cc:511] Master 127.8.64.190:39865 requested a full tablet report, sending...
I20260924 00:55:57.588207 9580 ts_manager.cc:194] Registered new tserver with Master: e4f058a495e44e7cbcc73bee4d367c7d (127.8.64.131:40187)
I20260924 00:55:57.589200 8450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 00:55:57.589232 9580 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.131@KRBTEST.COM'} at 127.8.64.131:41007
Sep 24 00:55:57 dist-test-slave-mmhs krb5kdc[9536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211357, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 24 00:55:57 dist-test-slave-mmhs krb5kdc[9536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211357, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:55:57.637614 9677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:57.623328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43666 (local address 127.8.64.190:39865)
0924 00:55:57.623517 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:57.623524 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:57.624327 (+ 803us) server_negotiation.cc:407] Connection header received
0924 00:55:57.624545 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:57.624573 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:57.624680 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:57.624792 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:55:57.626605 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:57.627463 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:57.628702 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:57.629125 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:57.629221 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:57.632873 (+ 3652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:55:57.632878 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:55:57.632882 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:55:57.632914 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:55:57.635323 (+ 2409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:57.635966 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:57.635972 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:57.635974 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:57.636118 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:57.636490 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:57.636494 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:57.636496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:57.636766 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:55:57.636943 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:57.637186 (+ 243us) server_negotiation.cc:299] Negotiation successful
0924 00:55:57.637351 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260924 00:55:57.641948 9580 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43666:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:55:57.648994 9580 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:55:57.685132 10086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:57.660819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56606 (local address 127.8.64.129:43265)
0924 00:55:57.674913 (+ 14094us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:57.674926 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:57.675699 (+ 773us) server_negotiation.cc:407] Connection header received
0924 00:55:57.675914 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:57.675925 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:57.676239 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:57.676426 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:55:57.677479 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:57.679576 (+ 2097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:57.682284 (+ 2708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:57.683767 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:57.683936 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:57.684143 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:57.684629 (+ 486us) server_negotiation.cc:299] Negotiation successful
0924 00:55:57.684805 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13849,"thread_start_us":13687,"threads_started":1}
I20260924 00:55:57.685181 10084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:57.660514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52366 (local address 127.8.64.130:46619)
0924 00:55:57.675198 (+ 14684us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:57.675204 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:57.675229 (+ 25us) server_negotiation.cc:407] Connection header received
0924 00:55:57.675306 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:57.675311 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:57.675521 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:57.675751 (+ 230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:55:57.676470 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:57.678221 (+ 1751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:57.682372 (+ 4151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:57.683709 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:57.683810 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:57.684032 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:57.684712 (+ 680us) server_negotiation.cc:299] Negotiation successful
0924 00:55:57.684876 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14536,"thread_start_us":14426,"threads_started":1}
I20260924 00:55:57.685340 10085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:57.660638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:43265 (local address 127.0.0.1:56606)
0924 00:55:57.675556 (+ 14918us) negotiation.cc:107] Waiting for socket to connect
0924 00:55:57.675597 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 00:55:57.675746 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:55:57.676650 (+ 904us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:55:57.676680 (+ 30us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:55:57.676713 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:55:57.677206 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:57.677222 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:57.679731 (+ 2509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:57.679736 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:57.682126 (+ 2390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:57.682138 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:57.683998 (+ 1860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:57.684005 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:57.684291 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:57.684409 (+ 118us) client_negotiation.cc:769] Sending connection context
0924 00:55:57.684565 (+ 156us) client_negotiation.cc:240] Negotiation successful
0924 00:55:57.684877 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14832,"mutex_wait_us":170,"thread_start_us":15221,"threads_started":1}
I20260924 00:55:57.685858 10083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:57.660323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.130:46619 (local address 127.0.0.1:52366)
0924 00:55:57.673392 (+ 13069us) negotiation.cc:107] Waiting for socket to connect
0924 00:55:57.673460 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0924 00:55:57.673670 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:55:57.675758 (+ 2088us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:55:57.675766 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:55:57.675796 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:55:57.676289 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:57.676307 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:57.678370 (+ 2063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:57.678377 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:57.682118 (+ 3741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:57.682131 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:57.684284 (+ 2153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:57.684291 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:57.684510 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:57.684554 (+ 44us) client_negotiation.cc:769] Sending connection context
0924 00:55:57.685415 (+ 861us) client_negotiation.cc:240] Negotiation successful
0924 00:55:57.685553 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12709,"mutex_wait_us":55,"thread_start_us":12565,"threads_started":1}
I20260924 00:55:57.689164 10088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:57.676302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.131:40187 (local address 127.0.0.1:49230)
0924 00:55:57.679386 (+ 3084us) negotiation.cc:107] Waiting for socket to connect
0924 00:55:57.679408 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0924 00:55:57.679549 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:55:57.680273 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:55:57.680281 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:55:57.680307 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:55:57.682369 (+ 2062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:57.682380 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:57.684256 (+ 1876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:57.684272 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:57.686555 (+ 2283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:57.686579 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:57.688391 (+ 1812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:57.688398 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:57.688564 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:57.688595 (+ 31us) client_negotiation.cc:769] Sending connection context
0924 00:55:57.688683 (+ 88us) client_negotiation.cc:240] Negotiation successful
0924 00:55:57.688806 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2942,"mutex_wait_us":18,"thread_start_us":2780,"threads_started":1}
I20260924 00:55:57.689224 10087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:57.676532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49230 (local address 127.8.64.131:40187)
0924 00:55:57.678913 (+ 2381us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:57.678921 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:57.679525 (+ 604us) server_negotiation.cc:407] Connection header received
0924 00:55:57.679748 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:57.679758 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:57.680006 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:57.680203 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:55:57.682576 (+ 2373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:57.684086 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:57.686748 (+ 2662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:57.688185 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:57.688318 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:57.688534 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:57.688765 (+ 231us) server_negotiation.cc:299] Negotiation successful
0924 00:55:57.688922 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2223,"thread_start_us":2045,"threads_started":1}
I20260924 00:55:57.689499 9719 tablet_service.cc:1467] Processing CreateTablet for tablet a8c52cd3145c4ceeb79cbcfb8c49500d (DEFAULT_TABLE table=test-table [id=c16e65d29e224e1c871c33953cb656d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:55:57.689527 9860 tablet_service.cc:1467] Processing CreateTablet for tablet a8c52cd3145c4ceeb79cbcfb8c49500d (DEFAULT_TABLE table=test-table [id=c16e65d29e224e1c871c33953cb656d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:55:57.690984 9719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8c52cd3145c4ceeb79cbcfb8c49500d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:55:57.691011 9860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8c52cd3145c4ceeb79cbcfb8c49500d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:55:57.692926 10000 tablet_service.cc:1467] Processing CreateTablet for tablet a8c52cd3145c4ceeb79cbcfb8c49500d (DEFAULT_TABLE table=test-table [id=c16e65d29e224e1c871c33953cb656d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:55:57.694367 10000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8c52cd3145c4ceeb79cbcfb8c49500d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:55:57.702236 10089 tablet_bootstrap.cc:492] T a8c52cd3145c4ceeb79cbcfb8c49500d P e73cb1b3c0c74cecafa9b41b52628f77: Bootstrap starting.
I20260924 00:55:57.705072 10089 tablet_bootstrap.cc:654] T a8c52cd3145c4ceeb79cbcfb8c49500d P e73cb1b3c0c74cecafa9b41b52628f77: Neither blocks nor log segments found. Creating new log.
I20260924 00:55:57.705780 10090 tablet_bootstrap.cc:492] T a8c52cd3145c4ceeb79cbcfb8c49500d P d06b687eed2a433781313535a31c682a: Bootstrap starting.
I20260924 00:55:57.706561 10089 log.cc:824] T a8c52cd3145c4ceeb79cbcfb8c49500d P e73cb1b3c0c74cecafa9b41b52628f77: Log is configured to *not* fsync() on all Append() calls
I20260924 00:55:57.708907 10092 tablet_bootstrap.cc:492] T a8c52cd3145c4ceeb79cbcfb8c49500d P e4f058a495e44e7cbcc73bee4d367c7d: Bootstrap starting.
I20260924 00:55:57.709300 10090 tablet_bootstrap.cc:654] T a8c52cd3145c4ceeb79cbcfb8c49500d P d06b687eed2a433781313535a31c682a: Neither blocks nor log segments found. Creating new log.
I20260924 00:55:57.710681 10090 log.cc:824] T a8c52cd3145c4ceeb79cbcfb8c49500d P d06b687eed2a433781313535a31c682a: Log is configured to *not* fsync() on all Append() calls
I20260924 00:55:57.710817 10089 tablet_bootstrap.cc:492] T a8c52cd3145c4ceeb79cbcfb8c49500d P e73cb1b3c0c74cecafa9b41b52628f77: No bootstrap required, opened a new log
I20260924 00:55:57.711123 10089 ts_tablet_manager.cc:1397] T a8c52cd3145c4ceeb79cbcfb8c49500d P e73cb1b3c0c74cecafa9b41b52628f77: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260924 00:55:57.712214 10092 tablet_bootstrap.cc:654] T a8c52cd3145c4ceeb79cbcfb8c49500d P e4f058a495e44e7cbcc73bee4d367c7d: Neither blocks nor log segments found. Creating new log.
I20260924 00:55:57.713150 10092 log.cc:824] T a8c52cd3145c4ceeb79cbcfb8c49500d P e4f058a495e44e7cbcc73bee4d367c7d: Log is configured to *not* fsync() on all Append() calls
I20260924 00:55:57.713639 10090 tablet_bootstrap.cc:492] T a8c52cd3145c4ceeb79cbcfb8c49500d P d06b687eed2a433781313535a31c682a: No bootstrap required, opened a new log
I20260924 00:55:57.713945 10090 ts_tablet_manager.cc:1397] T a8c52cd3145c4ceeb79cbcfb8c49500d P d06b687eed2a433781313535a31c682a: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.008s
I20260924 00:55:57.715531 10092 tablet_bootstrap.cc:492] T a8c52cd3145c4ceeb79cbcfb8c49500d P e4f058a495e44e7cbcc73bee4d367c7d: No bootstrap required, opened a new log
I20260924 00:55:57.715803 10092 ts_tablet_manager.cc:1397] T a8c52cd3145c4ceeb79cbcfb8c49500d P e4f058a495e44e7cbcc73bee4d367c7d: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260924 00:55:57.721684 10089 raft_consensus.cc:348] T a8c52cd3145c4ceeb79cbcfb8c49500d P e73cb1b3c0c74cecafa9b41b52628f77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46619 } } peers { permanent_uuid: "d06b687eed2a433781313535a31c682a" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 43265 } } peers { permanent_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 40187 } }
I20260924 00:55:57.722470 10089 raft_consensus.cc:374] T a8c52cd3145c4ceeb79cbcfb8c49500d P e73cb1b3c0c74cecafa9b41b52628f77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:55:57.722648 10089 raft_consensus.cc:729] T a8c52cd3145c4ceeb79cbcfb8c49500d P e73cb1b3c0c74cecafa9b41b52628f77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e73cb1b3c0c74cecafa9b41b52628f77, State: Initialized, Role: FOLLOWER
I20260924 00:55:57.723749 10089 consensus_queue.cc:260] T a8c52cd3145c4ceeb79cbcfb8c49500d P e73cb1b3c0c74cecafa9b41b52628f77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46619 } } peers { permanent_uuid: "d06b687eed2a433781313535a31c682a" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 43265 } } peers { permanent_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 40187 } }
I20260924 00:55:57.724025 10092 raft_consensus.cc:348] T a8c52cd3145c4ceeb79cbcfb8c49500d P e4f058a495e44e7cbcc73bee4d367c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46619 } } peers { permanent_uuid: "d06b687eed2a433781313535a31c682a" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 43265 } } peers { permanent_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 40187 } }
I20260924 00:55:57.724961 10092 raft_consensus.cc:374] T a8c52cd3145c4ceeb79cbcfb8c49500d P e4f058a495e44e7cbcc73bee4d367c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:55:57.724677 10090 raft_consensus.cc:348] T a8c52cd3145c4ceeb79cbcfb8c49500d P d06b687eed2a433781313535a31c682a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46619 } } peers { permanent_uuid: "d06b687eed2a433781313535a31c682a" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 43265 } } peers { permanent_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 40187 } }
I20260924 00:55:57.725102 10090 raft_consensus.cc:374] T a8c52cd3145c4ceeb79cbcfb8c49500d P d06b687eed2a433781313535a31c682a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:55:57.725260 10090 raft_consensus.cc:729] T a8c52cd3145c4ceeb79cbcfb8c49500d P d06b687eed2a433781313535a31c682a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d06b687eed2a433781313535a31c682a, State: Initialized, Role: FOLLOWER
I20260924 00:55:57.725371 9930 heartbeater.cc:503] Master 127.8.64.190:39865 was elected leader, sending a full tablet report...
I20260924 00:55:57.725890 10090 consensus_queue.cc:260] T a8c52cd3145c4ceeb79cbcfb8c49500d P d06b687eed2a433781313535a31c682a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46619 } } peers { permanent_uuid: "d06b687eed2a433781313535a31c682a" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 43265 } } peers { permanent_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 40187 } }
I20260924 00:55:57.726138 10089 ts_tablet_manager.cc:1428] T a8c52cd3145c4ceeb79cbcfb8c49500d P e73cb1b3c0c74cecafa9b41b52628f77: Time spent starting tablet: real 0.015s user 0.011s sys 0.001s
I20260924 00:55:57.725116 10092 raft_consensus.cc:729] T a8c52cd3145c4ceeb79cbcfb8c49500d P e4f058a495e44e7cbcc73bee4d367c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4f058a495e44e7cbcc73bee4d367c7d, State: Initialized, Role: FOLLOWER
I20260924 00:55:57.727499 10090 ts_tablet_manager.cc:1428] T a8c52cd3145c4ceeb79cbcfb8c49500d P d06b687eed2a433781313535a31c682a: Time spent starting tablet: real 0.013s user 0.011s sys 0.001s
I20260924 00:55:57.727365 10092 consensus_queue.cc:260] T a8c52cd3145c4ceeb79cbcfb8c49500d P e4f058a495e44e7cbcc73bee4d367c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46619 } } peers { permanent_uuid: "d06b687eed2a433781313535a31c682a" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 43265 } } peers { permanent_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 40187 } }
I20260924 00:55:57.731240 10070 heartbeater.cc:503] Master 127.8.64.190:39865 was elected leader, sending a full tablet report...
I20260924 00:55:57.731886 10092 ts_tablet_manager.cc:1428] T a8c52cd3145c4ceeb79cbcfb8c49500d P e4f058a495e44e7cbcc73bee4d367c7d: Time spent starting tablet: real 0.016s user 0.007s sys 0.007s
W20260924 00:55:57.811614 9791 tablet.cc:2369] T a8c52cd3145c4ceeb79cbcfb8c49500d P d06b687eed2a433781313535a31c682a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 00:55:57.820683 9931 tablet.cc:2369] T a8c52cd3145c4ceeb79cbcfb8c49500d P e73cb1b3c0c74cecafa9b41b52628f77: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:55:57.829299 10096 raft_consensus.cc:482] T a8c52cd3145c4ceeb79cbcfb8c49500d P d06b687eed2a433781313535a31c682a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:55:57.829589 10096 raft_consensus.cc:504] T a8c52cd3145c4ceeb79cbcfb8c49500d P d06b687eed2a433781313535a31c682a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46619 } } peers { permanent_uuid: "d06b687eed2a433781313535a31c682a" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 43265 } } peers { permanent_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 40187 } }
I20260924 00:55:57.831297 10096 leader_election.cc:288] T a8c52cd3145c4ceeb79cbcfb8c49500d P d06b687eed2a433781313535a31c682a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e73cb1b3c0c74cecafa9b41b52628f77 (127.8.64.130:46619), e4f058a495e44e7cbcc73bee4d367c7d (127.8.64.131:40187)
W20260924 00:55:57.834916 10071 tablet.cc:2369] T a8c52cd3145c4ceeb79cbcfb8c49500d P e4f058a495e44e7cbcc73bee4d367c7d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:55:57.839394 10084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:57.831924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:51843 (local address 127.8.64.130:46619)
0924 00:55:57.832373 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:57.832380 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:57.832895 (+ 515us) server_negotiation.cc:407] Connection header received
0924 00:55:57.833085 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:57.833091 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:57.833168 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:57.833256 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:55:57.834076 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:57.835227 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:57.837589 (+ 2362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:57.838406 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:57.838590 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:57.838664 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:57.838937 (+ 273us) server_negotiation.cc:299] Negotiation successful
0924 00:55:57.839065 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275}
I20260924 00:55:57.839406 10087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:57.831910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:35973 (local address 127.8.64.131:40187)
0924 00:55:57.832159 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:57.832169 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:57.832745 (+ 576us) server_negotiation.cc:407] Connection header received
0924 00:55:57.832997 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:57.833004 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:57.833085 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:57.833186 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:55:57.834072 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:57.835227 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:57.837632 (+ 2405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:57.838521 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:57.838707 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:57.838780 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:57.839030 (+ 250us) server_negotiation.cc:299] Negotiation successful
0924 00:55:57.839158 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260924 00:55:57.839476 10098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:57.831680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.130:46619 (local address 127.8.64.129:51843)
0924 00:55:57.832813 (+ 1133us) negotiation.cc:107] Waiting for socket to connect
0924 00:55:57.832827 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 00:55:57.832927 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:55:57.833271 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:55:57.833274 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:55:57.833289 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:55:57.833924 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:57.833935 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:57.835420 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:57.835428 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:57.837422 (+ 1994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:57.837438 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:57.838609 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:57.838615 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:57.838775 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:57.838798 (+ 23us) client_negotiation.cc:769] Sending connection context
0924 00:55:57.838900 (+ 102us) client_negotiation.cc:240] Negotiation successful
0924 00:55:57.839026 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1026,"mutex_wait_us":115,"thread_start_us":482,"threads_started":1}
I20260924 00:55:57.839534 10099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:57.831620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.131:40187 (local address 127.8.64.129:35973)
0924 00:55:57.832591 (+ 971us) negotiation.cc:107] Waiting for socket to connect
0924 00:55:57.832631 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 00:55:57.832796 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:55:57.833239 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:55:57.833245 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:55:57.833271 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:55:57.833880 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:57.833896 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:57.835367 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:57.835372 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:57.837472 (+ 2100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:57.837486 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:57.838735 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:57.838742 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:57.838881 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:57.838902 (+ 21us) client_negotiation.cc:769] Sending connection context
0924 00:55:57.839007 (+ 105us) client_negotiation.cc:240] Negotiation successful
0924 00:55:57.839131 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":766,"mutex_wait_us":49,"thread_start_us":869,"threads_started":1}
I20260924 00:55:57.840471 10025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8c52cd3145c4ceeb79cbcfb8c49500d" candidate_uuid: "d06b687eed2a433781313535a31c682a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" is_pre_election: true
I20260924 00:55:57.840489 9880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8c52cd3145c4ceeb79cbcfb8c49500d" candidate_uuid: "d06b687eed2a433781313535a31c682a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" is_pre_election: true
I20260924 00:55:57.840945 10025 raft_consensus.cc:2454] T a8c52cd3145c4ceeb79cbcfb8c49500d P e4f058a495e44e7cbcc73bee4d367c7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d06b687eed2a433781313535a31c682a in term 0.
I20260924 00:55:57.840981 9880 raft_consensus.cc:2454] T a8c52cd3145c4ceeb79cbcfb8c49500d P e73cb1b3c0c74cecafa9b41b52628f77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d06b687eed2a433781313535a31c682a in term 0.
I20260924 00:55:57.841653 9669 leader_election.cc:302] T a8c52cd3145c4ceeb79cbcfb8c49500d P d06b687eed2a433781313535a31c682a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d06b687eed2a433781313535a31c682a, e4f058a495e44e7cbcc73bee4d367c7d; no voters:
I20260924 00:55:57.842049 10096 raft_consensus.cc:2781] T a8c52cd3145c4ceeb79cbcfb8c49500d P d06b687eed2a433781313535a31c682a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:55:57.842193 10096 raft_consensus.cc:482] T a8c52cd3145c4ceeb79cbcfb8c49500d P d06b687eed2a433781313535a31c682a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:55:57.842321 10096 raft_consensus.cc:3037] T a8c52cd3145c4ceeb79cbcfb8c49500d P d06b687eed2a433781313535a31c682a [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:55:57.844425 10096 raft_consensus.cc:504] T a8c52cd3145c4ceeb79cbcfb8c49500d P d06b687eed2a433781313535a31c682a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46619 } } peers { permanent_uuid: "d06b687eed2a433781313535a31c682a" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 43265 } } peers { permanent_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 40187 } }
I20260924 00:55:57.845001 10096 leader_election.cc:288] T a8c52cd3145c4ceeb79cbcfb8c49500d P d06b687eed2a433781313535a31c682a [CANDIDATE]: Term 1 election: Requested vote from peers e73cb1b3c0c74cecafa9b41b52628f77 (127.8.64.130:46619), e4f058a495e44e7cbcc73bee4d367c7d (127.8.64.131:40187)
I20260924 00:55:57.845461 10025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8c52cd3145c4ceeb79cbcfb8c49500d" candidate_uuid: "d06b687eed2a433781313535a31c682a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4f058a495e44e7cbcc73bee4d367c7d"
I20260924 00:55:57.845568 9880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8c52cd3145c4ceeb79cbcfb8c49500d" candidate_uuid: "d06b687eed2a433781313535a31c682a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e73cb1b3c0c74cecafa9b41b52628f77"
I20260924 00:55:57.845671 10025 raft_consensus.cc:3037] T a8c52cd3145c4ceeb79cbcfb8c49500d P e4f058a495e44e7cbcc73bee4d367c7d [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:55:57.845783 9880 raft_consensus.cc:3037] T a8c52cd3145c4ceeb79cbcfb8c49500d P e73cb1b3c0c74cecafa9b41b52628f77 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:55:57.848626 9880 raft_consensus.cc:2454] T a8c52cd3145c4ceeb79cbcfb8c49500d P e73cb1b3c0c74cecafa9b41b52628f77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d06b687eed2a433781313535a31c682a in term 1.
I20260924 00:55:57.848651 10025 raft_consensus.cc:2454] T a8c52cd3145c4ceeb79cbcfb8c49500d P e4f058a495e44e7cbcc73bee4d367c7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d06b687eed2a433781313535a31c682a in term 1.
I20260924 00:55:57.849092 9672 leader_election.cc:302] T a8c52cd3145c4ceeb79cbcfb8c49500d P d06b687eed2a433781313535a31c682a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d06b687eed2a433781313535a31c682a, e73cb1b3c0c74cecafa9b41b52628f77; no voters:
I20260924 00:55:57.849464 10096 raft_consensus.cc:2781] T a8c52cd3145c4ceeb79cbcfb8c49500d P d06b687eed2a433781313535a31c682a [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:55:57.850020 10096 raft_consensus.cc:686] T a8c52cd3145c4ceeb79cbcfb8c49500d P d06b687eed2a433781313535a31c682a [term 1 LEADER]: Becoming Leader. State: Replica: d06b687eed2a433781313535a31c682a, State: Running, Role: LEADER
I20260924 00:55:57.850561 10096 consensus_queue.cc:237] T a8c52cd3145c4ceeb79cbcfb8c49500d P d06b687eed2a433781313535a31c682a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46619 } } peers { permanent_uuid: "d06b687eed2a433781313535a31c682a" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 43265 } } peers { permanent_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 40187 } }
I20260924 00:55:57.854893 9581 catalog_manager.cc:5495] T a8c52cd3145c4ceeb79cbcfb8c49500d P d06b687eed2a433781313535a31c682a reported cstate change: term changed from 0 to 1, leader changed from <none> to d06b687eed2a433781313535a31c682a (127.8.64.129). New cstate: current_term: 1 leader_uuid: "d06b687eed2a433781313535a31c682a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d06b687eed2a433781313535a31c682a" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 43265 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 40187 } health_report { overall_health: UNKNOWN } } }
I20260924 00:55:57.914100 10086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:57.908567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56614 (local address 127.8.64.129:43265)
0924 00:55:57.908773 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:57.908781 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:57.908800 (+ 19us) server_negotiation.cc:407] Connection header received
0924 00:55:57.909135 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:57.909139 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:57.909227 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:57.909369 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:55:57.909736 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:57.910496 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:57.911957 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:57.912353 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:57.912489 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:57.912999 (+ 510us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:55:57.913479 (+ 480us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:55:57.913640 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:57.913751 (+ 111us) server_negotiation.cc:299] Negotiation successful
0924 00:55:57.913830 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260924 00:55:57.922355 9880 raft_consensus.cc:1261] T a8c52cd3145c4ceeb79cbcfb8c49500d P e73cb1b3c0c74cecafa9b41b52628f77 [term 1 FOLLOWER]: Refusing update from remote peer d06b687eed2a433781313535a31c682a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:55:57.922362 10025 raft_consensus.cc:1261] T a8c52cd3145c4ceeb79cbcfb8c49500d P e4f058a495e44e7cbcc73bee4d367c7d [term 1 FOLLOWER]: Refusing update from remote peer d06b687eed2a433781313535a31c682a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:55:57.923378 10100 consensus_queue.cc:1035] T a8c52cd3145c4ceeb79cbcfb8c49500d P d06b687eed2a433781313535a31c682a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:55:57.923840 10096 consensus_queue.cc:1035] T a8c52cd3145c4ceeb79cbcfb8c49500d P d06b687eed2a433781313535a31c682a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 40187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:55:57.954499 9581 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43666:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:55:57.954784 9581 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43666:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:55:57.958791 9581 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c3573e3a5c4e473a927e9723abed68de: Sending DeleteTablet for 3 replicas of tablet a8c52cd3145c4ceeb79cbcfb8c49500d
I20260924 00:55:57.959827 9719 tablet_service.cc:1514] Processing DeleteTablet for tablet a8c52cd3145c4ceeb79cbcfb8c49500d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:55:57 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:56606
I20260924 00:55:57.959859 9860 tablet_service.cc:1514] Processing DeleteTablet for tablet a8c52cd3145c4ceeb79cbcfb8c49500d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:55:57 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:52366
I20260924 00:55:57.962393 10000 tablet_service.cc:1514] Processing DeleteTablet for tablet a8c52cd3145c4ceeb79cbcfb8c49500d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:55:57 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:49230
I20260924 00:55:57.970832 10115 tablet_replica.cc:330] T a8c52cd3145c4ceeb79cbcfb8c49500d P e73cb1b3c0c74cecafa9b41b52628f77: stopping tablet replica
I20260924 00:55:57.970832 10116 tablet_replica.cc:330] T a8c52cd3145c4ceeb79cbcfb8c49500d P e4f058a495e44e7cbcc73bee4d367c7d: stopping tablet replica
I20260924 00:55:57.971289 10116 raft_consensus.cc:2229] T a8c52cd3145c4ceeb79cbcfb8c49500d P e4f058a495e44e7cbcc73bee4d367c7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:55:57.971321 10115 raft_consensus.cc:2229] T a8c52cd3145c4ceeb79cbcfb8c49500d P e73cb1b3c0c74cecafa9b41b52628f77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:55:57.971352 9581 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43666:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:55:57.971786 10116 raft_consensus.cc:2258] T a8c52cd3145c4ceeb79cbcfb8c49500d P e4f058a495e44e7cbcc73bee4d367c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:55:57.971786 10115 raft_consensus.cc:2258] T a8c52cd3145c4ceeb79cbcfb8c49500d P e73cb1b3c0c74cecafa9b41b52628f77 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260924 00:55:57.972039 9581 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:55:57.972590 10114 tablet_replica.cc:330] T a8c52cd3145c4ceeb79cbcfb8c49500d P d06b687eed2a433781313535a31c682a: stopping tablet replica
I20260924 00:55:57.972992 10114 raft_consensus.cc:2229] T a8c52cd3145c4ceeb79cbcfb8c49500d P d06b687eed2a433781313535a31c682a [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:55:57.975121 10116 ts_tablet_manager.cc:1905] T a8c52cd3145c4ceeb79cbcfb8c49500d P e4f058a495e44e7cbcc73bee4d367c7d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:55:57.975517 10115 ts_tablet_manager.cc:1905] T a8c52cd3145c4ceeb79cbcfb8c49500d P e73cb1b3c0c74cecafa9b41b52628f77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:55:57.978559 10114 raft_consensus.cc:2258] T a8c52cd3145c4ceeb79cbcfb8c49500d P d06b687eed2a433781313535a31c682a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:55:57.980916 10000 tablet_service.cc:1467] Processing CreateTablet for tablet e9c85a515c564f08a16478d12f7e586a (DEFAULT_TABLE table=test-table [id=c6bb82e1b4544a9dbd9ba6a1e4865502]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:55:57.981128 9860 tablet_service.cc:1467] Processing CreateTablet for tablet e9c85a515c564f08a16478d12f7e586a (DEFAULT_TABLE table=test-table [id=c6bb82e1b4544a9dbd9ba6a1e4865502]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:55:57.981439 10000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9c85a515c564f08a16478d12f7e586a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:55:57.981546 9860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9c85a515c564f08a16478d12f7e586a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:55:57.982632 9719 tablet_service.cc:1467] Processing CreateTablet for tablet e9c85a515c564f08a16478d12f7e586a (DEFAULT_TABLE table=test-table [id=c6bb82e1b4544a9dbd9ba6a1e4865502]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:55:57.983053 9719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9c85a515c564f08a16478d12f7e586a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:55:57.983616 10115 ts_tablet_manager.cc:1918] T a8c52cd3145c4ceeb79cbcfb8c49500d P e73cb1b3c0c74cecafa9b41b52628f77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:55:57.983750 10115 log.cc:1196] T a8c52cd3145c4ceeb79cbcfb8c49500d P e73cb1b3c0c74cecafa9b41b52628f77: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-1/wal/wals/a8c52cd3145c4ceeb79cbcfb8c49500d
I20260924 00:55:57.984262 10115 ts_tablet_manager.cc:1939] T a8c52cd3145c4ceeb79cbcfb8c49500d P e73cb1b3c0c74cecafa9b41b52628f77: Deleting consensus metadata
I20260924 00:55:57.986399 9568 catalog_manager.cc:4841] TS e73cb1b3c0c74cecafa9b41b52628f77 (127.8.64.130:46619): tablet a8c52cd3145c4ceeb79cbcfb8c49500d (table test-table [id=c16e65d29e224e1c871c33953cb656d6]) successfully deleted
I20260924 00:55:57.989179 10092 tablet_bootstrap.cc:492] T e9c85a515c564f08a16478d12f7e586a P e4f058a495e44e7cbcc73bee4d367c7d: Bootstrap starting.
I20260924 00:55:57.989234 10116 ts_tablet_manager.cc:1918] T a8c52cd3145c4ceeb79cbcfb8c49500d P e4f058a495e44e7cbcc73bee4d367c7d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:55:57.989379 10116 log.cc:1196] T a8c52cd3145c4ceeb79cbcfb8c49500d P e4f058a495e44e7cbcc73bee4d367c7d: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-2/wal/wals/a8c52cd3145c4ceeb79cbcfb8c49500d
I20260924 00:55:57.989862 10116 ts_tablet_manager.cc:1939] T a8c52cd3145c4ceeb79cbcfb8c49500d P e4f058a495e44e7cbcc73bee4d367c7d: Deleting consensus metadata
I20260924 00:55:57.990883 10092 tablet_bootstrap.cc:654] T e9c85a515c564f08a16478d12f7e586a P e4f058a495e44e7cbcc73bee4d367c7d: Neither blocks nor log segments found. Creating new log.
I20260924 00:55:57.991727 9565 catalog_manager.cc:4841] TS e4f058a495e44e7cbcc73bee4d367c7d (127.8.64.131:40187): tablet a8c52cd3145c4ceeb79cbcfb8c49500d (table test-table [id=c16e65d29e224e1c871c33953cb656d6]) successfully deleted
I20260924 00:55:57.993247 10092 tablet_bootstrap.cc:492] T e9c85a515c564f08a16478d12f7e586a P e4f058a495e44e7cbcc73bee4d367c7d: No bootstrap required, opened a new log
I20260924 00:55:57.993472 10092 ts_tablet_manager.cc:1397] T e9c85a515c564f08a16478d12f7e586a P e4f058a495e44e7cbcc73bee4d367c7d: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20260924 00:55:57.994097 10114 ts_tablet_manager.cc:1905] T a8c52cd3145c4ceeb79cbcfb8c49500d P d06b687eed2a433781313535a31c682a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:55:57.994524 10089 tablet_bootstrap.cc:492] T e9c85a515c564f08a16478d12f7e586a P e73cb1b3c0c74cecafa9b41b52628f77: Bootstrap starting.
I20260924 00:55:57.994763 10092 raft_consensus.cc:348] T e9c85a515c564f08a16478d12f7e586a P e4f058a495e44e7cbcc73bee4d367c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06b687eed2a433781313535a31c682a" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 43265 } } peers { permanent_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46619 } } peers { permanent_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 40187 } }
I20260924 00:55:57.994982 10092 raft_consensus.cc:374] T e9c85a515c564f08a16478d12f7e586a P e4f058a495e44e7cbcc73bee4d367c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:55:57.995100 10092 raft_consensus.cc:729] T e9c85a515c564f08a16478d12f7e586a P e4f058a495e44e7cbcc73bee4d367c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4f058a495e44e7cbcc73bee4d367c7d, State: Initialized, Role: FOLLOWER
I20260924 00:55:57.995502 10092 consensus_queue.cc:260] T e9c85a515c564f08a16478d12f7e586a P e4f058a495e44e7cbcc73bee4d367c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06b687eed2a433781313535a31c682a" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 43265 } } peers { permanent_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46619 } } peers { permanent_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 40187 } }
I20260924 00:55:57.996340 10089 tablet_bootstrap.cc:654] T e9c85a515c564f08a16478d12f7e586a P e73cb1b3c0c74cecafa9b41b52628f77: Neither blocks nor log segments found. Creating new log.
I20260924 00:55:57.998871 10092 ts_tablet_manager.cc:1428] T e9c85a515c564f08a16478d12f7e586a P e4f058a495e44e7cbcc73bee4d367c7d: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260924 00:55:58.003131 10114 ts_tablet_manager.cc:1918] T a8c52cd3145c4ceeb79cbcfb8c49500d P d06b687eed2a433781313535a31c682a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:55:58.003347 10114 log.cc:1196] T a8c52cd3145c4ceeb79cbcfb8c49500d P d06b687eed2a433781313535a31c682a: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-0/wal/wals/a8c52cd3145c4ceeb79cbcfb8c49500d
I20260924 00:55:58.003937 10114 ts_tablet_manager.cc:1939] T a8c52cd3145c4ceeb79cbcfb8c49500d P d06b687eed2a433781313535a31c682a: Deleting consensus metadata
I20260924 00:55:58.005780 9565 catalog_manager.cc:4841] TS d06b687eed2a433781313535a31c682a (127.8.64.129:43265): tablet a8c52cd3145c4ceeb79cbcfb8c49500d (table test-table [id=c16e65d29e224e1c871c33953cb656d6]) successfully deleted
I20260924 00:55:58.007249 10089 tablet_bootstrap.cc:492] T e9c85a515c564f08a16478d12f7e586a P e73cb1b3c0c74cecafa9b41b52628f77: No bootstrap required, opened a new log
I20260924 00:55:58.007527 10089 ts_tablet_manager.cc:1397] T e9c85a515c564f08a16478d12f7e586a P e73cb1b3c0c74cecafa9b41b52628f77: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.009s
I20260924 00:55:58.008253 10090 tablet_bootstrap.cc:492] T e9c85a515c564f08a16478d12f7e586a P d06b687eed2a433781313535a31c682a: Bootstrap starting.
I20260924 00:55:58.008615 10089 raft_consensus.cc:348] T e9c85a515c564f08a16478d12f7e586a P e73cb1b3c0c74cecafa9b41b52628f77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06b687eed2a433781313535a31c682a" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 43265 } } peers { permanent_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46619 } } peers { permanent_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 40187 } }
I20260924 00:55:58.008811 10089 raft_consensus.cc:374] T e9c85a515c564f08a16478d12f7e586a P e73cb1b3c0c74cecafa9b41b52628f77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:55:58.008914 10089 raft_consensus.cc:729] T e9c85a515c564f08a16478d12f7e586a P e73cb1b3c0c74cecafa9b41b52628f77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e73cb1b3c0c74cecafa9b41b52628f77, State: Initialized, Role: FOLLOWER
I20260924 00:55:58.009177 10089 consensus_queue.cc:260] T e9c85a515c564f08a16478d12f7e586a P e73cb1b3c0c74cecafa9b41b52628f77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06b687eed2a433781313535a31c682a" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 43265 } } peers { permanent_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46619 } } peers { permanent_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 40187 } }
I20260924 00:55:58.009945 10089 ts_tablet_manager.cc:1428] T e9c85a515c564f08a16478d12f7e586a P e73cb1b3c0c74cecafa9b41b52628f77: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260924 00:55:58.010427 10090 tablet_bootstrap.cc:654] T e9c85a515c564f08a16478d12f7e586a P d06b687eed2a433781313535a31c682a: Neither blocks nor log segments found. Creating new log.
I20260924 00:55:58.013123 10090 tablet_bootstrap.cc:492] T e9c85a515c564f08a16478d12f7e586a P d06b687eed2a433781313535a31c682a: No bootstrap required, opened a new log
I20260924 00:55:58.013332 10090 ts_tablet_manager.cc:1397] T e9c85a515c564f08a16478d12f7e586a P d06b687eed2a433781313535a31c682a: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.005s
I20260924 00:55:58.014390 10090 raft_consensus.cc:348] T e9c85a515c564f08a16478d12f7e586a P d06b687eed2a433781313535a31c682a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06b687eed2a433781313535a31c682a" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 43265 } } peers { permanent_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46619 } } peers { permanent_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 40187 } }
I20260924 00:55:58.014621 10090 raft_consensus.cc:374] T e9c85a515c564f08a16478d12f7e586a P d06b687eed2a433781313535a31c682a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:55:58.014721 10090 raft_consensus.cc:729] T e9c85a515c564f08a16478d12f7e586a P d06b687eed2a433781313535a31c682a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d06b687eed2a433781313535a31c682a, State: Initialized, Role: FOLLOWER
I20260924 00:55:58.014973 10090 consensus_queue.cc:260] T e9c85a515c564f08a16478d12f7e586a P d06b687eed2a433781313535a31c682a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06b687eed2a433781313535a31c682a" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 43265 } } peers { permanent_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46619 } } peers { permanent_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 40187 } }
I20260924 00:55:58.015620 10090 ts_tablet_manager.cc:1428] T e9c85a515c564f08a16478d12f7e586a P d06b687eed2a433781313535a31c682a: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260924 00:55:58.101413 10095 raft_consensus.cc:482] T e9c85a515c564f08a16478d12f7e586a P e73cb1b3c0c74cecafa9b41b52628f77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:55:58.101643 10095 raft_consensus.cc:504] T e9c85a515c564f08a16478d12f7e586a P e73cb1b3c0c74cecafa9b41b52628f77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06b687eed2a433781313535a31c682a" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 43265 } } peers { permanent_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46619 } } peers { permanent_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 40187 } }
I20260924 00:55:58.103358 10095 leader_election.cc:288] T e9c85a515c564f08a16478d12f7e586a P e73cb1b3c0c74cecafa9b41b52628f77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d06b687eed2a433781313535a31c682a (127.8.64.129:43265), e4f058a495e44e7cbcc73bee4d367c7d (127.8.64.131:40187)
I20260924 00:55:58.111284 10086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:58.103687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:38493 (local address 127.8.64.129:43265)
0924 00:55:58.103923 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:58.103933 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:58.104479 (+ 546us) server_negotiation.cc:407] Connection header received
0924 00:55:58.104715 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:58.104725 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:58.104834 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:58.104925 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:55:58.105896 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:58.107002 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:58.109315 (+ 2313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:58.110106 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:58.110301 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:58.110401 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:58.110680 (+ 279us) server_negotiation.cc:299] Negotiation successful
0924 00:55:58.110797 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260924 00:55:58.111389 10120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:58.103675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:43265 (local address 127.8.64.130:38493)
0924 00:55:58.104366 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0924 00:55:58.104387 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0924 00:55:58.104528 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:55:58.105186 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:55:58.105193 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:55:58.105229 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:55:58.105693 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:58.105709 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:58.107130 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:58.107139 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:58.109164 (+ 2025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:58.109175 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:58.110363 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:58.110370 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:58.110522 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:58.110546 (+ 24us) client_negotiation.cc:769] Sending connection context
0924 00:55:58.110658 (+ 112us) client_negotiation.cc:240] Negotiation successful
0924 00:55:58.110786 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"mutex_wait_us":28,"thread_start_us":433,"threads_started":1}
I20260924 00:55:58.112404 9744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9c85a515c564f08a16478d12f7e586a" candidate_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d06b687eed2a433781313535a31c682a" is_pre_election: true
I20260924 00:55:58.112839 9744 raft_consensus.cc:2454] T e9c85a515c564f08a16478d12f7e586a P d06b687eed2a433781313535a31c682a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e73cb1b3c0c74cecafa9b41b52628f77 in term 0.
I20260924 00:55:58.113438 9810 leader_election.cc:302] T e9c85a515c564f08a16478d12f7e586a P e73cb1b3c0c74cecafa9b41b52628f77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d06b687eed2a433781313535a31c682a, e73cb1b3c0c74cecafa9b41b52628f77; no voters:
I20260924 00:55:58.113739 10087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:58.104725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:46521 (local address 127.8.64.131:40187)
0924 00:55:58.104894 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:58.104901 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:58.106168 (+ 1267us) server_negotiation.cc:407] Connection header received
0924 00:55:58.106354 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:58.106360 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:58.106434 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:58.106533 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:55:58.107237 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:58.108348 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:58.111373 (+ 3025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:58.112566 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:58.112687 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:58.112765 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:58.113358 (+ 593us) server_negotiation.cc:299] Negotiation successful
0924 00:55:58.113504 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260924 00:55:58.113905 10095 raft_consensus.cc:2781] T e9c85a515c564f08a16478d12f7e586a P e73cb1b3c0c74cecafa9b41b52628f77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:55:58.113932 10121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:58.104532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.131:40187 (local address 127.8.64.130:46521)
0924 00:55:58.106040 (+ 1508us) negotiation.cc:107] Waiting for socket to connect
0924 00:55:58.106058 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0924 00:55:58.106189 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:55:58.106563 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:55:58.106568 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:55:58.106594 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:55:58.107002 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:58.107025 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:58.108520 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:58.108526 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:58.111184 (+ 2658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:58.111203 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:58.112946 (+ 1743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:58.112952 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:58.113146 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:58.113179 (+ 33us) client_negotiation.cc:769] Sending connection context
0924 00:55:58.113310 (+ 131us) client_negotiation.cc:240] Negotiation successful
0924 00:55:58.113441 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1375,"mutex_wait_us":1,"thread_start_us":1163,"threads_started":1}
I20260924 00:55:58.114022 10095 raft_consensus.cc:482] T e9c85a515c564f08a16478d12f7e586a P e73cb1b3c0c74cecafa9b41b52628f77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:55:58.114144 10095 raft_consensus.cc:3037] T e9c85a515c564f08a16478d12f7e586a P e73cb1b3c0c74cecafa9b41b52628f77 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:55:58.114609 10025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9c85a515c564f08a16478d12f7e586a" candidate_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" is_pre_election: true
I20260924 00:55:58.114831 10025 raft_consensus.cc:2454] T e9c85a515c564f08a16478d12f7e586a P e4f058a495e44e7cbcc73bee4d367c7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e73cb1b3c0c74cecafa9b41b52628f77 in term 0.
I20260924 00:55:58.116684 10095 raft_consensus.cc:504] T e9c85a515c564f08a16478d12f7e586a P e73cb1b3c0c74cecafa9b41b52628f77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06b687eed2a433781313535a31c682a" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 43265 } } peers { permanent_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46619 } } peers { permanent_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 40187 } }
I20260924 00:55:58.117242 10095 leader_election.cc:288] T e9c85a515c564f08a16478d12f7e586a P e73cb1b3c0c74cecafa9b41b52628f77 [CANDIDATE]: Term 1 election: Requested vote from peers d06b687eed2a433781313535a31c682a (127.8.64.129:43265), e4f058a495e44e7cbcc73bee4d367c7d (127.8.64.131:40187)
I20260924 00:55:58.117707 10025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9c85a515c564f08a16478d12f7e586a" candidate_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4f058a495e44e7cbcc73bee4d367c7d"
I20260924 00:55:58.117901 10025 raft_consensus.cc:3037] T e9c85a515c564f08a16478d12f7e586a P e4f058a495e44e7cbcc73bee4d367c7d [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:55:58.117882 9744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9c85a515c564f08a16478d12f7e586a" candidate_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d06b687eed2a433781313535a31c682a"
I20260924 00:55:58.118088 9744 raft_consensus.cc:3037] T e9c85a515c564f08a16478d12f7e586a P d06b687eed2a433781313535a31c682a [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:55:58.120877 10025 raft_consensus.cc:2454] T e9c85a515c564f08a16478d12f7e586a P e4f058a495e44e7cbcc73bee4d367c7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e73cb1b3c0c74cecafa9b41b52628f77 in term 1.
I20260924 00:55:58.121368 9810 leader_election.cc:302] T e9c85a515c564f08a16478d12f7e586a P e73cb1b3c0c74cecafa9b41b52628f77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e4f058a495e44e7cbcc73bee4d367c7d, e73cb1b3c0c74cecafa9b41b52628f77; no voters:
I20260924 00:55:58.121709 10095 raft_consensus.cc:2781] T e9c85a515c564f08a16478d12f7e586a P e73cb1b3c0c74cecafa9b41b52628f77 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:55:58.121809 9744 raft_consensus.cc:2454] T e9c85a515c564f08a16478d12f7e586a P d06b687eed2a433781313535a31c682a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e73cb1b3c0c74cecafa9b41b52628f77 in term 1.
I20260924 00:55:58.122475 10095 raft_consensus.cc:686] T e9c85a515c564f08a16478d12f7e586a P e73cb1b3c0c74cecafa9b41b52628f77 [term 1 LEADER]: Becoming Leader. State: Replica: e73cb1b3c0c74cecafa9b41b52628f77, State: Running, Role: LEADER
I20260924 00:55:58.123070 10095 consensus_queue.cc:237] T e9c85a515c564f08a16478d12f7e586a P e73cb1b3c0c74cecafa9b41b52628f77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06b687eed2a433781313535a31c682a" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 43265 } } peers { permanent_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46619 } } peers { permanent_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 40187 } }
I20260924 00:55:58.126770 9580 catalog_manager.cc:5495] T e9c85a515c564f08a16478d12f7e586a P e73cb1b3c0c74cecafa9b41b52628f77 reported cstate change: term changed from 0 to 1, leader changed from <none> to e73cb1b3c0c74cecafa9b41b52628f77 (127.8.64.130). New cstate: current_term: 1 leader_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06b687eed2a433781313535a31c682a" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 43265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 40187 } health_report { overall_health: UNKNOWN } } }
Sep 24 00:55:58 dist-test-slave-mmhs krb5kdc[9536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211355, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:55:58.193917 10129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:58.177049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43670 (local address 127.8.64.190:39865)
0924 00:55:58.177861 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:58.177873 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:58.178129 (+ 256us) server_negotiation.cc:407] Connection header received
0924 00:55:58.178400 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:58.178404 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:58.178466 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:58.178601 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:55:58.180145 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:58.180950 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:58.182901 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:58.183476 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:58.183582 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:58.187672 (+ 4090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:55:58.187677 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:55:58.187681 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:55:58.187708 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:55:58.190674 (+ 2966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:58.191645 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:58.191648 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:58.191651 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:58.191772 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:58.192615 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:58.192618 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:58.192621 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:58.192871 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:55:58.192976 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:58.193496 (+ 520us) server_negotiation.cc:299] Negotiation successful
0924 00:55:58.193667 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":317,"threads_started":1}
I20260924 00:55:58.194404 10130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:58.177088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:39865 (local address 127.0.0.1:43670)
0924 00:55:58.178014 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0924 00:55:58.178033 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0924 00:55:58.178186 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:55:58.178686 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:55:58.178691 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:55:58.179637 (+ 946us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:55:58.179979 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:58.179992 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:58.181084 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:58.181089 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:58.182771 (+ 1682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:58.182786 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:58.183677 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:58.183683 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:58.183851 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:58.184608 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:55:58.184631 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:55:58.187492 (+ 2861us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:55:58.190801 (+ 3309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:55:58.190815 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:55:58.190823 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:55:58.191522 (+ 699us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:55:58.191881 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:55:58.191887 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:55:58.191892 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:55:58.192499 (+ 607us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:55:58.192998 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:55:58.193006 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:55:58.193227 (+ 221us) client_negotiation.cc:769] Sending connection context
0924 00:55:58.193426 (+ 199us) client_negotiation.cc:240] Negotiation successful
0924 00:55:58.193852 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":788,"mutex_wait_us":108,"thread_start_us":610,"threads_started":1}
I20260924 00:55:58.199608 9580 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:43670:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260924 00:55:58.200675 9580 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:55:58.207821 9860 tablet_service.cc:1467] Processing CreateTablet for tablet f004208e612244aea9dcb15d65ed11a7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fb96b7a6dc944501b753bc1ea3d7fd6a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 00:55:58.207824 9719 tablet_service.cc:1467] Processing CreateTablet for tablet f004208e612244aea9dcb15d65ed11a7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fb96b7a6dc944501b753bc1ea3d7fd6a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 00:55:58.208016 10000 tablet_service.cc:1467] Processing CreateTablet for tablet f004208e612244aea9dcb15d65ed11a7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fb96b7a6dc944501b753bc1ea3d7fd6a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 00:55:58.208403 9860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f004208e612244aea9dcb15d65ed11a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:55:58.208403 9719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f004208e612244aea9dcb15d65ed11a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:55:58.208590 10000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f004208e612244aea9dcb15d65ed11a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:55:58.216459 10092 tablet_bootstrap.cc:492] T f004208e612244aea9dcb15d65ed11a7 P e4f058a495e44e7cbcc73bee4d367c7d: Bootstrap starting.
I20260924 00:55:58.216768 10089 tablet_bootstrap.cc:492] T f004208e612244aea9dcb15d65ed11a7 P e73cb1b3c0c74cecafa9b41b52628f77: Bootstrap starting.
I20260924 00:55:58.218570 10092 tablet_bootstrap.cc:654] T f004208e612244aea9dcb15d65ed11a7 P e4f058a495e44e7cbcc73bee4d367c7d: Neither blocks nor log segments found. Creating new log.
I20260924 00:55:58.218585 10090 tablet_bootstrap.cc:492] T f004208e612244aea9dcb15d65ed11a7 P d06b687eed2a433781313535a31c682a: Bootstrap starting.
I20260924 00:55:58.218667 10089 tablet_bootstrap.cc:654] T f004208e612244aea9dcb15d65ed11a7 P e73cb1b3c0c74cecafa9b41b52628f77: Neither blocks nor log segments found. Creating new log.
I20260924 00:55:58.220456 10090 tablet_bootstrap.cc:654] T f004208e612244aea9dcb15d65ed11a7 P d06b687eed2a433781313535a31c682a: Neither blocks nor log segments found. Creating new log.
I20260924 00:55:58.227293 10089 tablet_bootstrap.cc:492] T f004208e612244aea9dcb15d65ed11a7 P e73cb1b3c0c74cecafa9b41b52628f77: No bootstrap required, opened a new log
I20260924 00:55:58.227545 10089 ts_tablet_manager.cc:1397] T f004208e612244aea9dcb15d65ed11a7 P e73cb1b3c0c74cecafa9b41b52628f77: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260924 00:55:58.227761 10090 tablet_bootstrap.cc:492] T f004208e612244aea9dcb15d65ed11a7 P d06b687eed2a433781313535a31c682a: No bootstrap required, opened a new log
I20260924 00:55:58.227931 10090 ts_tablet_manager.cc:1397] T f004208e612244aea9dcb15d65ed11a7 P d06b687eed2a433781313535a31c682a: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.004s
I20260924 00:55:58.228277 10089 raft_consensus.cc:348] T f004208e612244aea9dcb15d65ed11a7 P e73cb1b3c0c74cecafa9b41b52628f77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06b687eed2a433781313535a31c682a" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 43265 } } peers { permanent_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46619 } } peers { permanent_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 40187 } }
I20260924 00:55:58.228497 10089 raft_consensus.cc:374] T f004208e612244aea9dcb15d65ed11a7 P e73cb1b3c0c74cecafa9b41b52628f77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:55:58.228575 10089 raft_consensus.cc:729] T f004208e612244aea9dcb15d65ed11a7 P e73cb1b3c0c74cecafa9b41b52628f77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e73cb1b3c0c74cecafa9b41b52628f77, State: Initialized, Role: FOLLOWER
I20260924 00:55:58.228664 10090 raft_consensus.cc:348] T f004208e612244aea9dcb15d65ed11a7 P d06b687eed2a433781313535a31c682a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06b687eed2a433781313535a31c682a" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 43265 } } peers { permanent_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46619 } } peers { permanent_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 40187 } }
I20260924 00:55:58.228909 10090 raft_consensus.cc:374] T f004208e612244aea9dcb15d65ed11a7 P d06b687eed2a433781313535a31c682a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:55:58.228822 10089 consensus_queue.cc:260] T f004208e612244aea9dcb15d65ed11a7 P e73cb1b3c0c74cecafa9b41b52628f77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06b687eed2a433781313535a31c682a" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 43265 } } peers { permanent_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46619 } } peers { permanent_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 40187 } }
I20260924 00:55:58.228994 10090 raft_consensus.cc:729] T f004208e612244aea9dcb15d65ed11a7 P d06b687eed2a433781313535a31c682a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d06b687eed2a433781313535a31c682a, State: Initialized, Role: FOLLOWER
I20260924 00:55:58.229214 10090 consensus_queue.cc:260] T f004208e612244aea9dcb15d65ed11a7 P d06b687eed2a433781313535a31c682a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06b687eed2a433781313535a31c682a" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 43265 } } peers { permanent_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46619 } } peers { permanent_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 40187 } }
I20260924 00:55:58.229426 10089 ts_tablet_manager.cc:1428] T f004208e612244aea9dcb15d65ed11a7 P e73cb1b3c0c74cecafa9b41b52628f77: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260924 00:55:58.229277 10095 tablet_replica.cc:439] T f004208e612244aea9dcb15d65ed11a7 P e73cb1b3c0c74cecafa9b41b52628f77: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06b687eed2a433781313535a31c682a" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 43265 } } peers { permanent_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46619 } } peers { permanent_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 40187 } } }
I20260924 00:55:58.229632 10095 tablet_replica.cc:442] T f004208e612244aea9dcb15d65ed11a7 P e73cb1b3c0c74cecafa9b41b52628f77: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 00:55:58.229847 10090 ts_tablet_manager.cc:1428] T f004208e612244aea9dcb15d65ed11a7 P d06b687eed2a433781313535a31c682a: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260924 00:55:58.229832 10109 tablet_replica.cc:439] T f004208e612244aea9dcb15d65ed11a7 P d06b687eed2a433781313535a31c682a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06b687eed2a433781313535a31c682a" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 43265 } } peers { permanent_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46619 } } peers { permanent_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 40187 } } }
I20260924 00:55:58.230432 10109 tablet_replica.cc:442] T f004208e612244aea9dcb15d65ed11a7 P d06b687eed2a433781313535a31c682a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 00:55:58.232854 10092 tablet_bootstrap.cc:492] T f004208e612244aea9dcb15d65ed11a7 P e4f058a495e44e7cbcc73bee4d367c7d: No bootstrap required, opened a new log
I20260924 00:55:58.233107 10092 ts_tablet_manager.cc:1397] T f004208e612244aea9dcb15d65ed11a7 P e4f058a495e44e7cbcc73bee4d367c7d: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.007s
I20260924 00:55:58.234103 10092 raft_consensus.cc:348] T f004208e612244aea9dcb15d65ed11a7 P e4f058a495e44e7cbcc73bee4d367c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06b687eed2a433781313535a31c682a" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 43265 } } peers { permanent_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46619 } } peers { permanent_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 40187 } }
I20260924 00:55:58.234278 10092 raft_consensus.cc:374] T f004208e612244aea9dcb15d65ed11a7 P e4f058a495e44e7cbcc73bee4d367c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:55:58.234401 10092 raft_consensus.cc:729] T f004208e612244aea9dcb15d65ed11a7 P e4f058a495e44e7cbcc73bee4d367c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4f058a495e44e7cbcc73bee4d367c7d, State: Initialized, Role: FOLLOWER
I20260924 00:55:58.234624 10092 consensus_queue.cc:260] T f004208e612244aea9dcb15d65ed11a7 P e4f058a495e44e7cbcc73bee4d367c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06b687eed2a433781313535a31c682a" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 43265 } } peers { permanent_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46619 } } peers { permanent_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 40187 } }
I20260924 00:55:58.235164 10092 ts_tablet_manager.cc:1428] T f004208e612244aea9dcb15d65ed11a7 P e4f058a495e44e7cbcc73bee4d367c7d: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260924 00:55:58.235023 10097 tablet_replica.cc:439] T f004208e612244aea9dcb15d65ed11a7 P e4f058a495e44e7cbcc73bee4d367c7d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06b687eed2a433781313535a31c682a" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 43265 } } peers { permanent_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46619 } } peers { permanent_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 40187 } } }
I20260924 00:55:58.235368 10097 tablet_replica.cc:442] T f004208e612244aea9dcb15d65ed11a7 P e4f058a495e44e7cbcc73bee4d367c7d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 00:55:58.260022 10097 raft_consensus.cc:482] T f004208e612244aea9dcb15d65ed11a7 P e4f058a495e44e7cbcc73bee4d367c7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:55:58.260222 10097 raft_consensus.cc:504] T f004208e612244aea9dcb15d65ed11a7 P e4f058a495e44e7cbcc73bee4d367c7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06b687eed2a433781313535a31c682a" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 43265 } } peers { permanent_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46619 } } peers { permanent_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 40187 } }
I20260924 00:55:58.261771 10097 leader_election.cc:288] T f004208e612244aea9dcb15d65ed11a7 P e4f058a495e44e7cbcc73bee4d367c7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d06b687eed2a433781313535a31c682a (127.8.64.129:43265), e73cb1b3c0c74cecafa9b41b52628f77 (127.8.64.130:46619)
I20260924 00:55:58.270514 10086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:58.262482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:38639 (local address 127.8.64.129:43265)
0924 00:55:58.262820 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:58.262829 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:58.263651 (+ 822us) server_negotiation.cc:407] Connection header received
0924 00:55:58.263894 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:58.263898 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:58.263956 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:58.264055 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:55:58.264661 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:58.266016 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:58.268065 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:58.269452 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:58.269626 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:58.269727 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:58.270044 (+ 317us) server_negotiation.cc:299] Negotiation successful
0924 00:55:58.270202 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174}
I20260924 00:55:58.270540 10134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:58.261987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:43265 (local address 127.8.64.131:38639)
0924 00:55:58.263510 (+ 1523us) negotiation.cc:107] Waiting for socket to connect
0924 00:55:58.263535 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0924 00:55:58.263726 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:55:58.264096 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:55:58.264100 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:55:58.264131 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:55:58.264439 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:58.264455 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:58.266163 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:58.266170 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:58.267881 (+ 1711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:58.267894 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:58.269653 (+ 1759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:58.269659 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:58.269846 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:58.269882 (+ 36us) client_negotiation.cc:769] Sending connection context
0924 00:55:58.269994 (+ 112us) client_negotiation.cc:240] Negotiation successful
0924 00:55:58.270136 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1219,"thread_start_us":1024,"threads_started":1}
I20260924 00:55:58.271533 10084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:58.262356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:39607 (local address 127.8.64.130:46619)
0924 00:55:58.262615 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:58.262624 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:58.263833 (+ 1209us) server_negotiation.cc:407] Connection header received
0924 00:55:58.264107 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:58.264113 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:58.264188 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:58.264302 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:55:58.265425 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:58.266523 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:58.269192 (+ 2669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:58.270370 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:58.270518 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:58.270610 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:58.271124 (+ 514us) server_negotiation.cc:299] Negotiation successful
0924 00:55:58.271248 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260924 00:55:58.271428 9744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f004208e612244aea9dcb15d65ed11a7" candidate_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d06b687eed2a433781313535a31c682a" is_pre_election: true
I20260924 00:55:58.271600 10135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:58.262107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.130:46619 (local address 127.8.64.131:39607)
0924 00:55:58.263726 (+ 1619us) negotiation.cc:107] Waiting for socket to connect
0924 00:55:58.263745 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0924 00:55:58.263933 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:55:58.264321 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:55:58.264325 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:55:58.264354 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:55:58.264772 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:58.264793 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:58.266705 (+ 1912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:58.266710 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:58.269000 (+ 2290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:58.269017 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:58.270781 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:58.270786 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:58.270963 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:58.270999 (+ 36us) client_negotiation.cc:769] Sending connection context
0924 00:55:58.271095 (+ 96us) client_negotiation.cc:240] Negotiation successful
0924 00:55:58.271221 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1480,"thread_start_us":1300,"threads_started":1}
I20260924 00:55:58.271735 9744 raft_consensus.cc:2454] T f004208e612244aea9dcb15d65ed11a7 P d06b687eed2a433781313535a31c682a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4f058a495e44e7cbcc73bee4d367c7d in term 0.
I20260924 00:55:58.272429 9880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f004208e612244aea9dcb15d65ed11a7" candidate_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" is_pre_election: true
I20260924 00:55:58.272526 9950 leader_election.cc:302] T f004208e612244aea9dcb15d65ed11a7 P e4f058a495e44e7cbcc73bee4d367c7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d06b687eed2a433781313535a31c682a, e4f058a495e44e7cbcc73bee4d367c7d; no voters:
I20260924 00:55:58.272948 10097 raft_consensus.cc:2781] T f004208e612244aea9dcb15d65ed11a7 P e4f058a495e44e7cbcc73bee4d367c7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:55:58.273082 10097 raft_consensus.cc:482] T f004208e612244aea9dcb15d65ed11a7 P e4f058a495e44e7cbcc73bee4d367c7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:55:58.273075 9880 raft_consensus.cc:2454] T f004208e612244aea9dcb15d65ed11a7 P e73cb1b3c0c74cecafa9b41b52628f77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4f058a495e44e7cbcc73bee4d367c7d in term 0.
I20260924 00:55:58.273200 10097 raft_consensus.cc:3037] T f004208e612244aea9dcb15d65ed11a7 P e4f058a495e44e7cbcc73bee4d367c7d [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:55:58.275462 10097 raft_consensus.cc:504] T f004208e612244aea9dcb15d65ed11a7 P e4f058a495e44e7cbcc73bee4d367c7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06b687eed2a433781313535a31c682a" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 43265 } } peers { permanent_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46619 } } peers { permanent_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 40187 } }
I20260924 00:55:58.276098 10097 leader_election.cc:288] T f004208e612244aea9dcb15d65ed11a7 P e4f058a495e44e7cbcc73bee4d367c7d [CANDIDATE]: Term 1 election: Requested vote from peers d06b687eed2a433781313535a31c682a (127.8.64.129:43265), e73cb1b3c0c74cecafa9b41b52628f77 (127.8.64.130:46619)
I20260924 00:55:58.276564 9744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f004208e612244aea9dcb15d65ed11a7" candidate_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d06b687eed2a433781313535a31c682a"
I20260924 00:55:58.276691 9880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f004208e612244aea9dcb15d65ed11a7" candidate_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e73cb1b3c0c74cecafa9b41b52628f77"
I20260924 00:55:58.276815 9744 raft_consensus.cc:3037] T f004208e612244aea9dcb15d65ed11a7 P d06b687eed2a433781313535a31c682a [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:55:58.276894 9880 raft_consensus.cc:3037] T f004208e612244aea9dcb15d65ed11a7 P e73cb1b3c0c74cecafa9b41b52628f77 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:55:58.280049 9880 raft_consensus.cc:2454] T f004208e612244aea9dcb15d65ed11a7 P e73cb1b3c0c74cecafa9b41b52628f77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4f058a495e44e7cbcc73bee4d367c7d in term 1.
I20260924 00:55:58.280040 9744 raft_consensus.cc:2454] T f004208e612244aea9dcb15d65ed11a7 P d06b687eed2a433781313535a31c682a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4f058a495e44e7cbcc73bee4d367c7d in term 1.
I20260924 00:55:58.280450 9953 leader_election.cc:302] T f004208e612244aea9dcb15d65ed11a7 P e4f058a495e44e7cbcc73bee4d367c7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e4f058a495e44e7cbcc73bee4d367c7d, e73cb1b3c0c74cecafa9b41b52628f77; no voters:
I20260924 00:55:58.280908 10097 raft_consensus.cc:2781] T f004208e612244aea9dcb15d65ed11a7 P e4f058a495e44e7cbcc73bee4d367c7d [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:55:58.281420 10097 raft_consensus.cc:686] T f004208e612244aea9dcb15d65ed11a7 P e4f058a495e44e7cbcc73bee4d367c7d [term 1 LEADER]: Becoming Leader. State: Replica: e4f058a495e44e7cbcc73bee4d367c7d, State: Running, Role: LEADER
I20260924 00:55:58.282328 10097 consensus_queue.cc:237] T f004208e612244aea9dcb15d65ed11a7 P e4f058a495e44e7cbcc73bee4d367c7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06b687eed2a433781313535a31c682a" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 43265 } } peers { permanent_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46619 } } peers { permanent_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 40187 } }
I20260924 00:55:58.284698 10136 tablet_replica.cc:439] T f004208e612244aea9dcb15d65ed11a7 P e4f058a495e44e7cbcc73bee4d367c7d: TxnStatusTablet state changed. Reason: New leader e4f058a495e44e7cbcc73bee4d367c7d. Latest consensus state: current_term: 1 leader_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06b687eed2a433781313535a31c682a" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 43265 } } peers { permanent_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46619 } } peers { permanent_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 40187 } } }
I20260924 00:55:58.285040 10136 tablet_replica.cc:442] T f004208e612244aea9dcb15d65ed11a7 P e4f058a495e44e7cbcc73bee4d367c7d: This TxnStatusTablet replica's current role is: LEADER
I20260924 00:55:58.285738 10138 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260924 00:55:58.285969 10138 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260924 00:55:58.286048 9581 catalog_manager.cc:5495] T f004208e612244aea9dcb15d65ed11a7 P e4f058a495e44e7cbcc73bee4d367c7d reported cstate change: term changed from 0 to 1, leader changed from <none> to e4f058a495e44e7cbcc73bee4d367c7d (127.8.64.131). New cstate: current_term: 1 leader_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06b687eed2a433781313535a31c682a" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 43265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 40187 } health_report { overall_health: HEALTHY } } }
I20260924 00:55:58.314692 10130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:58.307011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.131:40187 (local address 127.0.0.1:49238)
0924 00:55:58.307320 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0924 00:55:58.307341 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0924 00:55:58.307516 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:55:58.308127 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:55:58.308132 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:55:58.308145 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 00:55:58.308359 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:58.308371 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:58.309827 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:58.309835 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:58.311457 (+ 1622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:58.311472 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:58.312615 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:58.312622 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:58.312815 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:58.312931 (+ 116us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 00:55:58.314051 (+ 1120us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 00:55:58.314085 (+ 34us) client_negotiation.cc:769] Sending connection context
0924 00:55:58.314184 (+ 99us) client_negotiation.cc:240] Negotiation successful
0924 00:55:58.314366 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167}
I20260924 00:55:58.314708 10087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:58.307066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49238 (local address 127.8.64.131:40187)
0924 00:55:58.307254 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:58.307260 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:58.307424 (+ 164us) server_negotiation.cc:407] Connection header received
0924 00:55:58.307818 (+ 394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:58.307825 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:58.307938 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:58.308091 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:55:58.308510 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:58.309680 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:58.311620 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:58.312331 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:58.312472 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:58.313177 (+ 705us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:55:58.313917 (+ 740us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:55:58.314072 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:58.314229 (+ 157us) server_negotiation.cc:299] Negotiation successful
0924 00:55:58.314363 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260924 00:55:58.319625 9880 raft_consensus.cc:1261] T f004208e612244aea9dcb15d65ed11a7 P e73cb1b3c0c74cecafa9b41b52628f77 [term 1 FOLLOWER]: Refusing update from remote peer e4f058a495e44e7cbcc73bee4d367c7d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:55:58.319988 9744 raft_consensus.cc:1261] T f004208e612244aea9dcb15d65ed11a7 P d06b687eed2a433781313535a31c682a [term 1 FOLLOWER]: Refusing update from remote peer e4f058a495e44e7cbcc73bee4d367c7d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:55:58.320451 10136 consensus_queue.cc:1035] T f004208e612244aea9dcb15d65ed11a7 P e4f058a495e44e7cbcc73bee4d367c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:55:58.320817 10097 consensus_queue.cc:1035] T f004208e612244aea9dcb15d65ed11a7 P e4f058a495e44e7cbcc73bee4d367c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d06b687eed2a433781313535a31c682a" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 43265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:55:58.324765 10109 tablet_replica.cc:439] T f004208e612244aea9dcb15d65ed11a7 P d06b687eed2a433781313535a31c682a: TxnStatusTablet state changed. Reason: New leader e4f058a495e44e7cbcc73bee4d367c7d. Latest consensus state: current_term: 1 leader_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06b687eed2a433781313535a31c682a" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 43265 } } peers { permanent_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46619 } } peers { permanent_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 40187 } } }
I20260924 00:55:58.325088 10109 tablet_replica.cc:442] T f004208e612244aea9dcb15d65ed11a7 P d06b687eed2a433781313535a31c682a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 00:55:58.331826 10136 tablet_replica.cc:439] T f004208e612244aea9dcb15d65ed11a7 P e4f058a495e44e7cbcc73bee4d367c7d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06b687eed2a433781313535a31c682a" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 43265 } } peers { permanent_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46619 } } peers { permanent_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 40187 } } }
I20260924 00:55:58.332098 10136 tablet_replica.cc:442] T f004208e612244aea9dcb15d65ed11a7 P e4f058a495e44e7cbcc73bee4d367c7d: This TxnStatusTablet replica's current role is: LEADER
I20260924 00:55:58.332279 10104 mvcc.cc:205] Tried to move back new op lower bound from 7332705723668733952 to 7332705723527741440. Current Snapshot: MvccSnapshot[applied={T|T < 7332705723668733952}]
I20260924 00:55:58.333737 10097 tablet_replica.cc:439] T f004208e612244aea9dcb15d65ed11a7 P e4f058a495e44e7cbcc73bee4d367c7d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06b687eed2a433781313535a31c682a" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 43265 } } peers { permanent_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46619 } } peers { permanent_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 40187 } } }
I20260924 00:55:58.334061 10097 tablet_replica.cc:442] T f004208e612244aea9dcb15d65ed11a7 P e4f058a495e44e7cbcc73bee4d367c7d: This TxnStatusTablet replica's current role is: LEADER
I20260924 00:55:58.336256 10095 tablet_replica.cc:439] T f004208e612244aea9dcb15d65ed11a7 P e73cb1b3c0c74cecafa9b41b52628f77: TxnStatusTablet state changed. Reason: New leader e4f058a495e44e7cbcc73bee4d367c7d. Latest consensus state: current_term: 1 leader_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06b687eed2a433781313535a31c682a" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 43265 } } peers { permanent_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46619 } } peers { permanent_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 40187 } } }
I20260924 00:55:58.336548 10095 tablet_replica.cc:442] T f004208e612244aea9dcb15d65ed11a7 P e73cb1b3c0c74cecafa9b41b52628f77: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 00:55:58.339726 10109 tablet_replica.cc:439] T f004208e612244aea9dcb15d65ed11a7 P d06b687eed2a433781313535a31c682a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06b687eed2a433781313535a31c682a" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 43265 } } peers { permanent_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46619 } } peers { permanent_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 40187 } } }
I20260924 00:55:58.339982 10109 tablet_replica.cc:442] T f004208e612244aea9dcb15d65ed11a7 P d06b687eed2a433781313535a31c682a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 00:55:58.345321 10095 tablet_replica.cc:439] T f004208e612244aea9dcb15d65ed11a7 P e73cb1b3c0c74cecafa9b41b52628f77: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06b687eed2a433781313535a31c682a" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 43265 } } peers { permanent_uuid: "e73cb1b3c0c74cecafa9b41b52628f77" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46619 } } peers { permanent_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 40187 } } }
I20260924 00:55:58.345613 10095 tablet_replica.cc:442] T f004208e612244aea9dcb15d65ed11a7 P e73cb1b3c0c74cecafa9b41b52628f77: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 00:55:58.353796 10084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:58.344475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52374 (local address 127.8.64.130:46619)
0924 00:55:58.344659 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:58.344665 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:58.344683 (+ 18us) server_negotiation.cc:407] Connection header received
0924 00:55:58.344779 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:58.344784 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:58.344881 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:58.345028 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:55:58.345824 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:58.348214 (+ 2390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:58.350639 (+ 2425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:58.351285 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:58.351415 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:58.352057 (+ 642us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:55:58.352801 (+ 744us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:55:58.352933 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:58.353174 (+ 241us) server_negotiation.cc:299] Negotiation successful
0924 00:55:58.353277 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260924 00:55:58.373787 10087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:58.366170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:59009 (local address 127.8.64.131:40187)
0924 00:55:58.366384 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:58.366390 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:58.367082 (+ 692us) server_negotiation.cc:407] Connection header received
0924 00:55:58.367351 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:58.367357 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:58.367455 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:58.367560 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:55:58.368134 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:58.369304 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:58.371530 (+ 2226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:58.372268 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:58.372407 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:58.372916 (+ 509us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:55:58.373053 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:55:58.373152 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:58.373380 (+ 228us) server_negotiation.cc:299] Negotiation successful
0924 00:55:58.373468 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260924 00:55:58.373807 10145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:58.366233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.131:40187 (local address 127.8.64.130:59009)
0924 00:55:58.366946 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0924 00:55:58.366966 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0924 00:55:58.367191 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:55:58.367588 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:55:58.367595 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:55:58.367608 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 00:55:58.367942 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:58.367954 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:58.369448 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:58.369453 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:58.371295 (+ 1842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:58.371310 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:58.372459 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:58.372465 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:58.372641 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:58.372716 (+ 75us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 00:55:58.373161 (+ 445us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 00:55:58.373172 (+ 11us) client_negotiation.cc:769] Sending connection context
0924 00:55:58.373259 (+ 87us) client_negotiation.cc:240] Negotiation successful
0924 00:55:58.373405 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":407,"threads_started":1}
I20260924 00:55:58.386684 10025 raft_consensus.cc:1261] T e9c85a515c564f08a16478d12f7e586a P e4f058a495e44e7cbcc73bee4d367c7d [term 1 FOLLOWER]: Refusing update from remote peer e73cb1b3c0c74cecafa9b41b52628f77: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:55:58.387545 10122 consensus_queue.cc:1035] T e9c85a515c564f08a16478d12f7e586a P e73cb1b3c0c74cecafa9b41b52628f77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4f058a495e44e7cbcc73bee4d367c7d" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 40187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:55:58.390955 9744 raft_consensus.cc:1261] T e9c85a515c564f08a16478d12f7e586a P d06b687eed2a433781313535a31c682a [term 1 FOLLOWER]: Refusing update from remote peer e73cb1b3c0c74cecafa9b41b52628f77: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:55:58.391539 10122 consensus_queue.cc:1035] T e9c85a515c564f08a16478d12f7e586a P e73cb1b3c0c74cecafa9b41b52628f77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d06b687eed2a433781313535a31c682a" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 43265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:55:58.407235 10105 mvcc.cc:205] Tried to move back new op lower bound from 7332705723943518208 to 7332705722876469248. Current Snapshot: MvccSnapshot[applied={T|T < 7332705723943518208}]
I20260924 00:55:58.411834 10103 mvcc.cc:205] Tried to move back new op lower bound from 7332705723943518208 to 7332705722876469248. Current Snapshot: MvccSnapshot[applied={T|T < 7332705723943518208}]
I20260924 00:55:58.453745 10084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:58.440333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:49169 (local address 127.8.64.130:46619)
0924 00:55:58.440501 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:58.440508 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:58.445812 (+ 5304us) server_negotiation.cc:407] Connection header received
0924 00:55:58.446152 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:58.446158 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:58.446250 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:58.446348 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:55:58.447690 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:58.448865 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:58.451316 (+ 2451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:58.452002 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:58.452142 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:58.452912 (+ 770us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:55:58.453061 (+ 149us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:55:58.453167 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:58.453365 (+ 198us) server_negotiation.cc:299] Negotiation successful
0924 00:55:58.453447 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260924 00:55:58.454618 10150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:58.440386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.130:46619 (local address 127.8.64.131:49169)
0924 00:55:58.445686 (+ 5300us) negotiation.cc:107] Waiting for socket to connect
0924 00:55:58.445724 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0924 00:55:58.445932 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:55:58.446549 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:55:58.446553 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:55:58.446565 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 00:55:58.447454 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:58.447470 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:58.449048 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:58.449053 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:58.451155 (+ 2102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:55:58.451170 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:55:58.452277 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:55:58.452284 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:55:58.452565 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:58.452675 (+ 110us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 00:55:58.453187 (+ 512us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 00:55:58.453200 (+ 13us) client_negotiation.cc:769] Sending connection context
0924 00:55:58.453989 (+ 789us) client_negotiation.cc:240] Negotiation successful
0924 00:55:58.454145 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5040,"thread_start_us":5949,"threads_started":1}
I20260924 00:55:58.509395 9580 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43666:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:55:58.509671 9580 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43666:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:55:58.513089 9580 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c3573e3a5c4e473a927e9723abed68de: Sending DeleteTablet for 3 replicas of tablet e9c85a515c564f08a16478d12f7e586a
I20260924 00:55:58.513835 9719 tablet_service.cc:1514] Processing DeleteTablet for tablet e9c85a515c564f08a16478d12f7e586a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:55:58 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:56606
I20260924 00:55:58.514075 9860 tablet_service.cc:1514] Processing DeleteTablet for tablet e9c85a515c564f08a16478d12f7e586a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:55:58 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:52366
I20260924 00:55:58.515213 10000 tablet_service.cc:1514] Processing DeleteTablet for tablet e9c85a515c564f08a16478d12f7e586a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:55:58 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:49230
I20260924 00:55:58.523295 10157 tablet_replica.cc:330] T e9c85a515c564f08a16478d12f7e586a P e4f058a495e44e7cbcc73bee4d367c7d: stopping tablet replica
I20260924 00:55:58.524062 10157 raft_consensus.cc:2229] T e9c85a515c564f08a16478d12f7e586a P e4f058a495e44e7cbcc73bee4d367c7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:55:58.525262 10157 raft_consensus.cc:2258] T e9c85a515c564f08a16478d12f7e586a P e4f058a495e44e7cbcc73bee4d367c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:55:58.527052 10157 ts_tablet_manager.cc:1905] T e9c85a515c564f08a16478d12f7e586a P e4f058a495e44e7cbcc73bee4d367c7d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:55:58.532407 10156 tablet_replica.cc:330] T e9c85a515c564f08a16478d12f7e586a P e73cb1b3c0c74cecafa9b41b52628f77: stopping tablet replica
I20260924 00:55:58.532704 10156 raft_consensus.cc:2229] T e9c85a515c564f08a16478d12f7e586a P e73cb1b3c0c74cecafa9b41b52628f77 [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:55:58.533728 10156 raft_consensus.cc:2258] T e9c85a515c564f08a16478d12f7e586a P e73cb1b3c0c74cecafa9b41b52628f77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:55:58.534344 10155 tablet_replica.cc:330] T e9c85a515c564f08a16478d12f7e586a P d06b687eed2a433781313535a31c682a: stopping tablet replica
I20260924 00:55:58.534582 10155 raft_consensus.cc:2229] T e9c85a515c564f08a16478d12f7e586a P d06b687eed2a433781313535a31c682a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:55:58.534732 10156 ts_tablet_manager.cc:1905] T e9c85a515c564f08a16478d12f7e586a P e73cb1b3c0c74cecafa9b41b52628f77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:55:58.535406 10155 raft_consensus.cc:2258] T e9c85a515c564f08a16478d12f7e586a P d06b687eed2a433781313535a31c682a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:55:58.536474 10155 ts_tablet_manager.cc:1905] T e9c85a515c564f08a16478d12f7e586a P d06b687eed2a433781313535a31c682a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:55:58.542177 10156 ts_tablet_manager.cc:1918] T e9c85a515c564f08a16478d12f7e586a P e73cb1b3c0c74cecafa9b41b52628f77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 00:55:58.542692 10156 log.cc:1196] T e9c85a515c564f08a16478d12f7e586a P e73cb1b3c0c74cecafa9b41b52628f77: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-1/wal/wals/e9c85a515c564f08a16478d12f7e586a
I20260924 00:55:58.543533 10156 ts_tablet_manager.cc:1939] T e9c85a515c564f08a16478d12f7e586a P e73cb1b3c0c74cecafa9b41b52628f77: Deleting consensus metadata
I20260924 00:55:58.547248 10157 ts_tablet_manager.cc:1918] T e9c85a515c564f08a16478d12f7e586a P e4f058a495e44e7cbcc73bee4d367c7d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 00:55:58.547466 9568 catalog_manager.cc:4841] TS e73cb1b3c0c74cecafa9b41b52628f77 (127.8.64.130:46619): tablet e9c85a515c564f08a16478d12f7e586a (table test-table [id=c6bb82e1b4544a9dbd9ba6a1e4865502]) successfully deleted
I20260924 00:55:58.547529 10157 log.cc:1196] T e9c85a515c564f08a16478d12f7e586a P e4f058a495e44e7cbcc73bee4d367c7d: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-2/wal/wals/e9c85a515c564f08a16478d12f7e586a
I20260924 00:55:58.548141 10157 ts_tablet_manager.cc:1939] T e9c85a515c564f08a16478d12f7e586a P e4f058a495e44e7cbcc73bee4d367c7d: Deleting consensus metadata
I20260924 00:55:58.548418 10155 ts_tablet_manager.cc:1918] T e9c85a515c564f08a16478d12f7e586a P d06b687eed2a433781313535a31c682a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 00:55:58.549214 10155 log.cc:1196] T e9c85a515c564f08a16478d12f7e586a P d06b687eed2a433781313535a31c682a: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790211343099518-8450-0/minicluster-data/ts-0/wal/wals/e9c85a515c564f08a16478d12f7e586a
I20260924 00:55:58.549324 9565 catalog_manager.cc:4841] TS e4f058a495e44e7cbcc73bee4d367c7d (127.8.64.131:40187): tablet e9c85a515c564f08a16478d12f7e586a (table test-table [id=c6bb82e1b4544a9dbd9ba6a1e4865502]) successfully deleted
I20260924 00:55:58.550786 10155 ts_tablet_manager.cc:1939] T e9c85a515c564f08a16478d12f7e586a P d06b687eed2a433781313535a31c682a: Deleting consensus metadata
I20260924 00:55:58.552220 9565 catalog_manager.cc:4841] TS d06b687eed2a433781313535a31c682a (127.8.64.129:43265): tablet e9c85a515c564f08a16478d12f7e586a (table test-table [id=c6bb82e1b4544a9dbd9ba6a1e4865502]) successfully deleted
Sep 24 00:55:58 dist-test-slave-mmhs krb5kdc[9536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211357, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.64.129@KRBTEST.COM
I20260924 00:55:58.563802 10086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:58.530768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56620 (local address 127.8.64.129:43265)
0924 00:55:58.530949 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:58.530957 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:58.535212 (+ 4255us) server_negotiation.cc:407] Connection header received
0924 00:55:58.536861 (+ 1649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:58.536867 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:58.536955 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:58.537036 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:55:58.544463 (+ 7427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:58.545532 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:58.549391 (+ 3859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:58.550043 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:58.550141 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:58.556057 (+ 5916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:55:58.556073 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:55:58.556094 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:55:58.556138 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:55:58.560225 (+ 4087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:58.561182 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:58.561192 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:58.561196 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:58.561362 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:58.561810 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:58.561816 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:58.561820 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:58.562249 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:55:58.562591 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:58.563165 (+ 574us) server_negotiation.cc:299] Negotiation successful
0924 00:55:58.563418 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260924 00:55:58.566229 9785 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56620
I20260924 00:55:58.604842 10129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:58.580752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43682 (local address 127.8.64.190:39865)
0924 00:55:58.580948 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:58.580954 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:58.587894 (+ 6940us) server_negotiation.cc:407] Connection header received
0924 00:55:58.588357 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:58.588363 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:58.588476 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:58.588550 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:55:58.590930 (+ 2380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:58.592131 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:58.593990 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:58.594674 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:58.594788 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:58.597853 (+ 3065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:55:58.597859 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:55:58.597864 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:55:58.597901 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:55:58.601618 (+ 3717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:58.602508 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:58.602514 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:58.602517 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:58.602654 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:58.603121 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:58.603127 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:58.603130 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:58.603474 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:55:58.603700 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:58.604306 (+ 606us) server_negotiation.cc:299] Negotiation successful
0924 00:55:58.604532 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260924 00:55:58.606441 9580 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43682
I20260924 00:55:58.610024 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 9653
I20260924 00:55:58.638118 10086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:58.613001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56626 (local address 127.8.64.129:43265)
0924 00:55:58.613177 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:58.613183 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:58.621332 (+ 8149us) server_negotiation.cc:407] Connection header received
0924 00:55:58.621816 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:58.621822 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:58.621911 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:58.621990 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:55:58.623854 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:58.624964 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:58.626879 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:58.627536 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:58.627641 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:58.630852 (+ 3211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:55:58.630859 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:55:58.630864 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:55:58.630906 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:55:58.634910 (+ 4004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:58.635765 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:58.635771 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:58.635775 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:58.635910 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:58.636334 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:58.636339 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:58.636343 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:58.636747 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:55:58.636981 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:58.637526 (+ 545us) server_negotiation.cc:299] Negotiation successful
0924 00:55:58.637760 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260924 00:55:58.639696 9785 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56626
W20260924 00:55:58.640326 8450 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 9653: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260924 00:55:58.640509 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 9653
I20260924 00:55:58.670946 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 9794
Sep 24 00:55:58 dist-test-slave-mmhs krb5kdc[9536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211357, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.64.130@KRBTEST.COM
I20260924 00:55:58.691553 10084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:58.671795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52390 (local address 127.8.64.130:46619)
0924 00:55:58.671986 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:58.671992 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:58.672123 (+ 131us) server_negotiation.cc:407] Connection header received
0924 00:55:58.672311 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:58.672316 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:58.672396 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:58.672512 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:55:58.674140 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:58.675366 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:58.677240 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:58.677886 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:58.677988 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:58.683676 (+ 5688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:55:58.683695 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:55:58.683714 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:55:58.683761 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:55:58.688051 (+ 4290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:58.688900 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:58.688911 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:58.688915 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:58.689117 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:58.689549 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:58.689555 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:58.689559 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:58.690067 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:55:58.690431 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:58.690695 (+ 264us) server_negotiation.cc:299] Negotiation successful
0924 00:55:58.690932 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260924 00:55:58.692261 9925 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52390
W20260924 00:55:58.692988 8450 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 9794: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260924 00:55:58.693186 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 9794
I20260924 00:55:58.719784 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 9934
Sep 24 00:55:58 dist-test-slave-mmhs krb5kdc[9536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211357, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.64.131@KRBTEST.COM
I20260924 00:55:58.737984 10087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:55:58.720655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49254 (local address 127.8.64.131:40187)
0924 00:55:58.720836 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0924 00:55:58.720841 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:55:58.720877 (+ 36us) server_negotiation.cc:407] Connection header received
0924 00:55:58.721088 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:55:58.721092 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:55:58.721156 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:55:58.721234 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:55:58.722869 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:58.723603 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:58.725379 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:55:58.725785 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:55:58.725879 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:55:58.731072 (+ 5193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:55:58.731083 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:55:58.731102 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:55:58.731134 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:55:58.734980 (+ 3846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:58.735783 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:58.735794 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:58.735797 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:58.735973 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:55:58.736319 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:55:58.736325 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:55:58.736329 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:55:58.736786 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:55:58.737126 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0924 00:55:58.737347 (+ 221us) server_negotiation.cc:299] Negotiation successful
0924 00:55:58.737587 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260924 00:55:58.738780 10065 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49254
W20260924 00:55:58.739405 8450 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 9934: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260924 00:55:58.739573 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 9934
I20260924 00:55:58.760144 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 9549
I20260924 00:55:58.761264 9622 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:55:58.874540 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 9549
2026-09-24T00:55:58Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4622 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:55:58 dist-test-slave-mmhs krb5kdc[10166](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:55:58 dist-test-slave-mmhs krb5kdc[10166](info): set up 2 sockets
Sep 24 00:55:58 dist-test-slave-mmhs krb5kdc[10166](info): commencing operation
krb5kdc: starting...
W20260924 00:56:00.040186 8450 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.005s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Sep 24 00:56:00 dist-test-slave-mmhs krb5kdc[10166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211360, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:56:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:56:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.64.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
I20260924 00:56:00.230414 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:44123
--webserver_interface=127.8.64.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:41059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:44123
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:56:00.553031 10179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:00.553382 10179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:00.553556 10179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:00.561187 10179 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:56:00.561349 10179 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260924 00:56:00.561451 10179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:00.561578 10179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:00.561672 10179 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:56:00.561790 10179 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:56:00.572118 10179 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:41059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:44123
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:44123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:00.573535 10179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:00.574986 10179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:00.586975 10187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:00.587067 10184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:00.587014 10185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:00.588212 10179 server_base.cc:1034] running on GCE node
I20260924 00:56:00.589007 10179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:00.590346 10179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:00.591578 10179 hybrid_clock.cc:648] HybridClock initialized: now 1790211360591509 us; error 46 us; skew 500 ppm
I20260924 00:56:00.591920 10179 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:00 dist-test-slave-mmhs krb5kdc[10166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211360, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:00.607736 10179 init.cc:377] Logged in from keytab as kudu/127.8.64.190@KRBTEST.COM (short username kudu)
I20260924 00:56:00.609345 10179 webserver.cc:460] Webserver started at http://127.8.64.190:40737/ using document root <none> and password file <none>
I20260924 00:56:00.610060 10179 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:00.610184 10179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:00.610546 10179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:56:00.613539 10179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "d0cc323515b340cbb1ea1249b47797e8"
format_stamp: "Formatted at 2026-09-24 00:56:00 on dist-test-slave-mmhs"
server_key: "20ce9c9c927b887ecc4b3329875fe0a6"
server_key_iv: "dd6003fd8d2c4946722c312dabb8c908"
server_key_version: "encryptionkey@0"
I20260924 00:56:00.614367 10179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "d0cc323515b340cbb1ea1249b47797e8"
format_stamp: "Formatted at 2026-09-24 00:56:00 on dist-test-slave-mmhs"
server_key: "20ce9c9c927b887ecc4b3329875fe0a6"
server_key_iv: "dd6003fd8d2c4946722c312dabb8c908"
server_key_version: "encryptionkey@0"
I20260924 00:56:00.619578 10179 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260924 00:56:00.623148 10194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:00.624552 10179 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260924 00:56:00.624742 10179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "d0cc323515b340cbb1ea1249b47797e8"
format_stamp: "Formatted at 2026-09-24 00:56:00 on dist-test-slave-mmhs"
server_key: "20ce9c9c927b887ecc4b3329875fe0a6"
server_key_iv: "dd6003fd8d2c4946722c312dabb8c908"
server_key_version: "encryptionkey@0"
I20260924 00:56:00.624959 10179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:00.642999 10179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:00.646976 10179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:00.647238 10179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:00.669380 10179 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.190:44123
I20260924 00:56:00.669405 10255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.190:44123 every 8 connection(s)
I20260924 00:56:00.670790 10179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 00:56:00.674475 10256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:00.677711 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 10179
I20260924 00:56:00.677942 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 00:56:00.678627 8450 external_mini_cluster.cc:1428] Setting key 0ae4b6b6b851a254e6611903ad75ca8c
I20260924 00:56:00.687548 10256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0cc323515b340cbb1ea1249b47797e8: Bootstrap starting.
I20260924 00:56:00.691087 10256 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0cc323515b340cbb1ea1249b47797e8: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:00.692044 10256 log.cc:824] T 00000000000000000000000000000000 P d0cc323515b340cbb1ea1249b47797e8: Log is configured to *not* fsync() on all Append() calls
Sep 24 00:56:00 dist-test-slave-mmhs krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211360, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:56:00.695114 10256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0cc323515b340cbb1ea1249b47797e8: No bootstrap required, opened a new log
I20260924 00:56:00.702942 10260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:00.680596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56752 (local address 127.8.64.190:44123)
0924 00:56:00.681370 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:00.681386 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:00.681448 (+ 62us) server_negotiation.cc:407] Connection header received
0924 00:56:00.683334 (+ 1886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:00.683367 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:00.683890 (+ 523us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:00.684331 (+ 441us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:00.685744 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:00.687689 (+ 1945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:00.689021 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:00.690028 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:00.690204 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:00.694406 (+ 4202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:00.694418 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:00.694436 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:00.694482 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:00.698483 (+ 4001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:00.699138 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:00.699147 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:00.699155 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:00.699352 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:00.699734 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:00.699739 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:00.699742 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:00.700379 (+ 637us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:00.700672 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:00.701091 (+ 419us) server_negotiation.cc:299] Negotiation successful
0924 00:56:00.701538 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":321,"threads_started":1}
I20260924 00:56:00.705725 10256 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0cc323515b340cbb1ea1249b47797e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0cc323515b340cbb1ea1249b47797e8" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44123 } }
I20260924 00:56:00.706103 10256 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0cc323515b340cbb1ea1249b47797e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:00.706183 10256 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0cc323515b340cbb1ea1249b47797e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0cc323515b340cbb1ea1249b47797e8, State: Initialized, Role: FOLLOWER
I20260924 00:56:00.706764 10256 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0cc323515b340cbb1ea1249b47797e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0cc323515b340cbb1ea1249b47797e8" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44123 } }
I20260924 00:56:00.706969 10256 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0cc323515b340cbb1ea1249b47797e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:56:00.707070 10256 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0cc323515b340cbb1ea1249b47797e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:56:00.707187 10256 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0cc323515b340cbb1ea1249b47797e8 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:00.709203 10256 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0cc323515b340cbb1ea1249b47797e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0cc323515b340cbb1ea1249b47797e8" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44123 } }
I20260924 00:56:00.709687 10256 leader_election.cc:302] T 00000000000000000000000000000000 P d0cc323515b340cbb1ea1249b47797e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0cc323515b340cbb1ea1249b47797e8; no voters:
I20260924 00:56:00.710486 10256 leader_election.cc:288] T 00000000000000000000000000000000 P d0cc323515b340cbb1ea1249b47797e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:56:00.710521 10262 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0cc323515b340cbb1ea1249b47797e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:56:00.711241 10262 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0cc323515b340cbb1ea1249b47797e8 [term 1 LEADER]: Becoming Leader. State: Replica: d0cc323515b340cbb1ea1249b47797e8, State: Running, Role: LEADER
I20260924 00:56:00.711831 10262 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0cc323515b340cbb1ea1249b47797e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0cc323515b340cbb1ea1249b47797e8" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44123 } }
I20260924 00:56:00.712577 10256 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0cc323515b340cbb1ea1249b47797e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:56:00.714717 10264 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0cc323515b340cbb1ea1249b47797e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0cc323515b340cbb1ea1249b47797e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0cc323515b340cbb1ea1249b47797e8" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44123 } } }
I20260924 00:56:00.715158 10264 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0cc323515b340cbb1ea1249b47797e8 [sys.catalog]: This master's current role is: LEADER
I20260924 00:56:00.714860 10263 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0cc323515b340cbb1ea1249b47797e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0cc323515b340cbb1ea1249b47797e8. Latest consensus state: current_term: 1 leader_uuid: "d0cc323515b340cbb1ea1249b47797e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0cc323515b340cbb1ea1249b47797e8" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44123 } } }
I20260924 00:56:00.715440 10263 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0cc323515b340cbb1ea1249b47797e8 [sys.catalog]: This master's current role is: LEADER
I20260924 00:56:00.717984 10269 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:56:00.723629 10269 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:56:00.731942 10269 catalog_manager.cc:1347] Generated new cluster ID: c0833b194106422da3243aab547dbcac
I20260924 00:56:00.732055 10269 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:56:00.746078 10269 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:56:00.747370 10269 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:56:00.756110 10269 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0cc323515b340cbb1ea1249b47797e8: Generated new TSK 0
I20260924 00:56:00.757185 10269 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.64.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab.
Entry for principal kudu/127.8.64.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab.
I20260924 00:56:00.810225 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.129:0
--local_ip_for_outbound_sockets=127.8.64.129
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:44123
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab
--principal=kudu/127.8.64.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:41059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:56:01.134919 10283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:01.135200 10283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:01.135298 10283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:01.142400 10283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:01.142513 10283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:01.142598 10283 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260924 00:56:01.142650 10283 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 00:56:01.142714 10283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.129
I20260924 00:56:01.152361 10283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:41059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab
--principal=kudu/127.8.64.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.64.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.64.190:44123
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.129
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:01.153553 10283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:01.154892 10283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:01.167969 10289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:01.168007 10291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:01.168197 10288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:01.168852 10283 server_base.cc:1034] running on GCE node
I20260924 00:56:01.169452 10283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:01.170397 10283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:01.171615 10283 hybrid_clock.cc:648] HybridClock initialized: now 1790211361171561 us; error 47 us; skew 500 ppm
I20260924 00:56:01.171924 10283 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:01 dist-test-slave-mmhs krb5kdc[10166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211361, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:01.175009 10283 init.cc:377] Logged in from keytab as kudu/127.8.64.129@KRBTEST.COM (short username kudu)
I20260924 00:56:01.176467 10283 webserver.cc:460] Webserver started at http://127.8.64.129:39673/ using document root <none> and password file <none>
I20260924 00:56:01.177083 10283 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:01.177182 10283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:01.177428 10283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:56:01.180248 10283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-0/data/instance:
uuid: "70748f17176b4448ad9cfbe5e1e7c33e"
format_stamp: "Formatted at 2026-09-24 00:56:01 on dist-test-slave-mmhs"
server_key: "80cda92e03232c66bd2c7c68d3064a25"
server_key_iv: "47d825f0ef411f250c1bf988b838633e"
server_key_version: "encryptionkey@0"
I20260924 00:56:01.180956 10283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance:
uuid: "70748f17176b4448ad9cfbe5e1e7c33e"
format_stamp: "Formatted at 2026-09-24 00:56:01 on dist-test-slave-mmhs"
server_key: "80cda92e03232c66bd2c7c68d3064a25"
server_key_iv: "47d825f0ef411f250c1bf988b838633e"
server_key_version: "encryptionkey@0"
I20260924 00:56:01.185787 10283 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260924 00:56:01.190981 10298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:01.192762 10283 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260924 00:56:01.192943 10283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-0/wal
uuid: "70748f17176b4448ad9cfbe5e1e7c33e"
format_stamp: "Formatted at 2026-09-24 00:56:01 on dist-test-slave-mmhs"
server_key: "80cda92e03232c66bd2c7c68d3064a25"
server_key_iv: "47d825f0ef411f250c1bf988b838633e"
server_key_version: "encryptionkey@0"
I20260924 00:56:01.193203 10283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:01.221527 10283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:01.225754 10283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:01.226045 10283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:01.228699 10283 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:56:01.228789 10283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:01.228952 10283 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:56:01.229055 10283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 00:56:01 dist-test-slave-mmhs krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211361, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:56:01.255443 10307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:01.229778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:56459 (local address 127.8.64.190:44123)
0924 00:56:01.230267 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:01.230274 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:01.231602 (+ 1328us) server_negotiation.cc:407] Connection header received
0924 00:56:01.234849 (+ 3247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:01.234855 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:01.234953 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:01.235108 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:01.237434 (+ 2326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:01.238566 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:01.241188 (+ 2622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:01.241834 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:01.242005 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:01.247423 (+ 5418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:01.247432 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:01.247448 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:01.247499 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:01.251616 (+ 4117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:01.252723 (+ 1107us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:01.252732 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:01.252738 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:01.252913 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:01.253354 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:01.253361 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:01.253367 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:01.253829 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:01.254012 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:01.254831 (+ 819us) server_negotiation.cc:299] Negotiation successful
0924 00:56:01.255053 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":245,"threads_started":1}
I20260924 00:56:01.256830 10308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:01.230509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44123 (local address 127.8.64.129:56459)
0924 00:56:01.231397 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:01.231464 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:01.234462 (+ 2998us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:01.235373 (+ 911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:01.235385 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:01.236244 (+ 859us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:01.237244 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:01.237264 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:01.238785 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:01.238789 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:01.240974 (+ 2185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:01.240992 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:01.242421 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:01.242427 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:01.242808 (+ 381us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:01.243624 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:01.243664 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:01.247142 (+ 3478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:01.251781 (+ 4639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:01.251791 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:01.251815 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:01.252571 (+ 756us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:01.253047 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:01.253051 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:01.253053 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:01.253244 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:01.253996 (+ 752us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:01.254002 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:01.254442 (+ 440us) client_negotiation.cc:769] Sending connection context
0924 00:56:01.254746 (+ 304us) client_negotiation.cc:240] Negotiation successful
0924 00:56:01.255293 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":359,"threads_started":1}
I20260924 00:56:01.278007 10283 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.129:46381
I20260924 00:56:01.278028 10419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.129:46381 every 8 connection(s)
I20260924 00:56:01.278548 10307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:01.263569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:36783 (local address 127.8.64.190:44123)
0924 00:56:01.263767 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:01.263775 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:01.264444 (+ 669us) server_negotiation.cc:407] Connection header received
0924 00:56:01.264684 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:01.264689 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:01.264774 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:01.264870 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:01.266200 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:01.267599 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:01.269648 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:01.270062 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:01.270202 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:01.272736 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:01.272742 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:01.272747 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:01.272784 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:01.275565 (+ 2781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:01.276471 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:01.276477 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:01.276480 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:01.276613 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:01.276995 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:01.277002 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:01.277005 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:01.277358 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:01.277508 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:01.277941 (+ 433us) server_negotiation.cc:299] Negotiation successful
0924 00:56:01.278179 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260924 00:56:01.278681 10391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:01.263558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44123 (local address 127.8.64.129:36783)
0924 00:56:01.264333 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:01.264358 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:01.264542 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:01.265062 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:01.265066 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:01.265745 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:01.266051 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:01.266059 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:01.267755 (+ 1696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:01.267763 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:01.269504 (+ 1741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:01.269516 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:01.270243 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:01.270250 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:01.270471 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:01.271176 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:01.271192 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:01.272550 (+ 1358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:01.275719 (+ 3169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:01.275728 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:01.275732 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:01.276351 (+ 619us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:01.276727 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:01.276732 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:01.276735 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:01.276877 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:01.277473 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:01.277478 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:01.277689 (+ 211us) client_negotiation.cc:769] Sending connection context
0924 00:56:01.277841 (+ 152us) client_negotiation.cc:240] Negotiation successful
0924 00:56:01.278212 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":373,"threads_started":1}
I20260924 00:56:01.279507 10283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
I20260924 00:56:01.281612 10420 heartbeater.cc:347] Connected to a master server at 127.8.64.190:44123
I20260924 00:56:01.281873 10420 heartbeater.cc:464] Registering TS with master...
I20260924 00:56:01.282704 10420 heartbeater.cc:511] Master 127.8.64.190:44123 requested a full tablet report, sending...
I20260924 00:56:01.284435 10211 ts_manager.cc:194] Registered new tserver with Master: 70748f17176b4448ad9cfbe5e1e7c33e (127.8.64.129:46381)
I20260924 00:56:01.285804 10211 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.129@KRBTEST.COM'} at 127.8.64.129:56459
I20260924 00:56:01.289565 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 10283
I20260924 00:56:01.289779 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance
I20260924 00:56:01.290446 8450 external_mini_cluster.cc:1428] Setting key aae783042909064c97065642f92c600f
WARNING: no policy specified for kudu/127.8.64.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab.
Entry for principal kudu/127.8.64.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab.
I20260924 00:56:01.333506 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.130:0
--local_ip_for_outbound_sockets=127.8.64.130
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:44123
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab
--principal=kudu/127.8.64.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:41059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:56:01.651120 10424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:01.651393 10424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:01.651494 10424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:01.658680 10424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:01.658790 10424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:01.658874 10424 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260924 00:56:01.658924 10424 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 00:56:01.658993 10424 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.130
I20260924 00:56:01.668998 10424 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:41059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab
--principal=kudu/127.8.64.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.64.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.64.190:44123
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.130
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:01.670270 10424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:01.671569 10424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:01.684499 10430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:01.684782 10432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:01.685283 10424 server_base.cc:1034] running on GCE node
W20260924 00:56:01.685359 10429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:01.686019 10424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:01.686937 10424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:01.688159 10424 hybrid_clock.cc:648] HybridClock initialized: now 1790211361688083 us; error 87 us; skew 500 ppm
I20260924 00:56:01.688465 10424 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:01 dist-test-slave-mmhs krb5kdc[10166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211361, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:01.691409 10424 init.cc:377] Logged in from keytab as kudu/127.8.64.130@KRBTEST.COM (short username kudu)
I20260924 00:56:01.692760 10424 webserver.cc:460] Webserver started at http://127.8.64.130:38227/ using document root <none> and password file <none>
I20260924 00:56:01.693416 10424 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:01.693564 10424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:01.693871 10424 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:56:01.696746 10424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-1/data/instance:
uuid: "e51782766430416cba83a564995c41ba"
format_stamp: "Formatted at 2026-09-24 00:56:01 on dist-test-slave-mmhs"
server_key: "65e318c35cefdf0721d9385cebb9801e"
server_key_iv: "2d43fd0a1948adf87e7f4ad3f678c7fc"
server_key_version: "encryptionkey@0"
I20260924 00:56:01.697497 10424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance:
uuid: "e51782766430416cba83a564995c41ba"
format_stamp: "Formatted at 2026-09-24 00:56:01 on dist-test-slave-mmhs"
server_key: "65e318c35cefdf0721d9385cebb9801e"
server_key_iv: "2d43fd0a1948adf87e7f4ad3f678c7fc"
server_key_version: "encryptionkey@0"
I20260924 00:56:01.702374 10424 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260924 00:56:01.705576 10439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:01.706925 10424 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260924 00:56:01.707103 10424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-1/wal
uuid: "e51782766430416cba83a564995c41ba"
format_stamp: "Formatted at 2026-09-24 00:56:01 on dist-test-slave-mmhs"
server_key: "65e318c35cefdf0721d9385cebb9801e"
server_key_iv: "2d43fd0a1948adf87e7f4ad3f678c7fc"
server_key_version: "encryptionkey@0"
I20260924 00:56:01.707311 10424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:01.727608 10424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:01.731616 10424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:01.731914 10424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:01.734493 10424 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:56:01.734635 10424 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:01.734761 10424 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:56:01.734848 10424 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 00:56:01 dist-test-slave-mmhs krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211361, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.130@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:56:01.758622 10307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:01.735345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:48049 (local address 127.8.64.190:44123)
0924 00:56:01.735531 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:01.735535 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:01.737000 (+ 1465us) server_negotiation.cc:407] Connection header received
0924 00:56:01.739257 (+ 2257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:01.739263 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:01.739388 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:01.739499 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:01.742079 (+ 2580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:01.743157 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:01.745913 (+ 2756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:01.746543 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:01.746687 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:01.751976 (+ 5289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:01.751982 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:01.751985 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:01.752018 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:01.755011 (+ 2993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:01.755766 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:01.755771 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:01.755775 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:01.755897 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:01.756439 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:01.756448 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:01.756464 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:01.756956 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:01.757146 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:01.757996 (+ 850us) server_negotiation.cc:299] Negotiation successful
0924 00:56:01.758232 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260924 00:56:01.760269 10448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:01.735970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44123 (local address 127.8.64.130:48049)
0924 00:56:01.736834 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:01.736878 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:01.738961 (+ 2083us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:01.739894 (+ 933us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:01.739910 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:01.740885 (+ 975us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:01.741866 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:01.741896 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:01.743386 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:01.743390 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:01.745691 (+ 2301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:01.745717 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:01.746793 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:01.746802 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:01.747059 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:01.747960 (+ 901us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:01.747982 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:01.751709 (+ 3727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:01.755160 (+ 3451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:01.755170 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:01.755195 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:01.755634 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:01.756015 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:01.756026 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:01.756029 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:01.756247 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:01.757112 (+ 865us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:01.757120 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:01.757611 (+ 491us) client_negotiation.cc:769] Sending connection context
0924 00:56:01.757913 (+ 302us) client_negotiation.cc:240] Negotiation successful
0924 00:56:01.758508 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"thread_start_us":373,"threads_started":1}
I20260924 00:56:01.783437 10307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:01.766554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:47893 (local address 127.8.64.190:44123)
0924 00:56:01.766759 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:01.766764 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:01.767524 (+ 760us) server_negotiation.cc:407] Connection header received
0924 00:56:01.767732 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:01.767737 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:01.767825 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:01.767941 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:01.769736 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:01.771045 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:01.773214 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:01.773882 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:01.774038 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:01.776293 (+ 2255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:01.776303 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:01.776311 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:01.776363 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:01.780395 (+ 4032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:01.781181 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:01.781190 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:01.781196 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:01.781386 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:01.781776 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:01.781781 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:01.781785 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:01.782274 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:01.782463 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:01.782843 (+ 380us) server_negotiation.cc:299] Negotiation successful
0924 00:56:01.783054 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260924 00:56:01.783530 10513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:01.766637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44123 (local address 127.8.64.130:47893)
0924 00:56:01.767418 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:01.767438 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:01.767581 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:01.768140 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:01.768148 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:01.768903 (+ 755us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:01.769512 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:01.769530 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:01.771221 (+ 1691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:01.771232 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:01.772970 (+ 1738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:01.772982 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:01.774052 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:01.774056 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:01.774176 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:01.774932 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:01.774962 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:01.776074 (+ 1112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:01.780536 (+ 4462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:01.780539 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:01.780542 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:01.781041 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:01.781509 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:01.781513 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:01.781515 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:01.781644 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:01.782429 (+ 785us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:01.782434 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:01.782615 (+ 181us) client_negotiation.cc:769] Sending connection context
0924 00:56:01.782758 (+ 143us) client_negotiation.cc:240] Negotiation successful
0924 00:56:01.783076 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":457,"threads_started":1}
I20260924 00:56:01.788475 10424 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.130:45425
I20260924 00:56:01.788520 10558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.130:45425 every 8 connection(s)
I20260924 00:56:01.789680 10424 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
I20260924 00:56:01.791682 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 10424
I20260924 00:56:01.791810 10559 heartbeater.cc:347] Connected to a master server at 127.8.64.190:44123
I20260924 00:56:01.791859 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance
I20260924 00:56:01.792044 10559 heartbeater.cc:464] Registering TS with master...
I20260924 00:56:01.792359 8450 external_mini_cluster.cc:1428] Setting key 4fc932e976c5f52d0bf31276c193aa34
I20260924 00:56:01.792718 10559 heartbeater.cc:511] Master 127.8.64.190:44123 requested a full tablet report, sending...
I20260924 00:56:01.794190 10210 ts_manager.cc:194] Registered new tserver with Master: e51782766430416cba83a564995c41ba (127.8.64.130:45425)
I20260924 00:56:01.795073 10210 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.130@KRBTEST.COM'} at 127.8.64.130:48049
WARNING: no policy specified for kudu/127.8.64.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab.
Entry for principal kudu/127.8.64.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab.
I20260924 00:56:01.833958 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.131:0
--local_ip_for_outbound_sockets=127.8.64.131
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:44123
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab
--principal=kudu/127.8.64.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:41059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:56:02.150038 10563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:02.150341 10563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:02.150471 10563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:02.157750 10563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:02.157859 10563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:02.157961 10563 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260924 00:56:02.158012 10563 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 00:56:02.158074 10563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.131
I20260924 00:56:02.167884 10563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:41059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab
--principal=kudu/127.8.64.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.64.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.64.190:44123
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.131
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:02.169044 10563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:02.170236 10563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:02.182717 10568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:02.182879 10569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:02.182843 10571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:02.183657 10563 server_base.cc:1034] running on GCE node
I20260924 00:56:02.184216 10563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:02.185045 10563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:02.186259 10563 hybrid_clock.cc:648] HybridClock initialized: now 1790211362186208 us; error 37 us; skew 500 ppm
I20260924 00:56:02.186589 10563 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:02 dist-test-slave-mmhs krb5kdc[10166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211362, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:02.189749 10563 init.cc:377] Logged in from keytab as kudu/127.8.64.131@KRBTEST.COM (short username kudu)
I20260924 00:56:02.191116 10563 webserver.cc:460] Webserver started at http://127.8.64.131:34395/ using document root <none> and password file <none>
I20260924 00:56:02.191721 10563 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:02.191818 10563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:02.192068 10563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:56:02.194871 10563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-2/data/instance:
uuid: "08a0bd05c3b04ce594713e51cec5e82a"
format_stamp: "Formatted at 2026-09-24 00:56:02 on dist-test-slave-mmhs"
server_key: "eec1caab2b0357f432b0c9b509f7ac4b"
server_key_iv: "22c0320f82539d2662c227934db0d121"
server_key_version: "encryptionkey@0"
I20260924 00:56:02.195567 10563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance:
uuid: "08a0bd05c3b04ce594713e51cec5e82a"
format_stamp: "Formatted at 2026-09-24 00:56:02 on dist-test-slave-mmhs"
server_key: "eec1caab2b0357f432b0c9b509f7ac4b"
server_key_iv: "22c0320f82539d2662c227934db0d121"
server_key_version: "encryptionkey@0"
I20260924 00:56:02.200820 10563 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260924 00:56:02.204226 10578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:02.205631 10563 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260924 00:56:02.205821 10563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-2/wal
uuid: "08a0bd05c3b04ce594713e51cec5e82a"
format_stamp: "Formatted at 2026-09-24 00:56:02 on dist-test-slave-mmhs"
server_key: "eec1caab2b0357f432b0c9b509f7ac4b"
server_key_iv: "22c0320f82539d2662c227934db0d121"
server_key_version: "encryptionkey@0"
I20260924 00:56:02.206029 10563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:02.224071 10563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:02.228550 10563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:02.228844 10563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:02.231971 10563 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:56:02.232107 10563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:02.232221 10563 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:56:02.232335 10563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 00:56:02 dist-test-slave-mmhs krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211362, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.131@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:56:02.257484 10587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:02.232976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44123 (local address 127.8.64.131:38619)
0924 00:56:02.233858 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:02.233905 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:02.235769 (+ 1864us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:02.236490 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:02.236499 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:02.237234 (+ 735us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:02.238151 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:02.238183 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.239781 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.239788 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:02.242172 (+ 2384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:02.242184 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.243338 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.243346 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:02.243583 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:02.244469 (+ 886us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:02.244491 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:02.247794 (+ 3303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:02.252455 (+ 4661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:02.252462 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:02.252474 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:02.253091 (+ 617us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:02.253590 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:02.253607 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:02.253613 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:02.253835 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:02.254539 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:02.254552 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:02.254926 (+ 374us) client_negotiation.cc:769] Sending connection context
0924 00:56:02.255322 (+ 396us) client_negotiation.cc:240] Negotiation successful
0924 00:56:02.255924 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":363,"threads_started":1}
I20260924 00:56:02.258615 10307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:02.232297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:38619 (local address 127.8.64.190:44123)
0924 00:56:02.232486 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:02.232494 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:02.234049 (+ 1555us) server_negotiation.cc:407] Connection header received
0924 00:56:02.236045 (+ 1996us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:02.236050 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:02.236145 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:02.236253 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:02.238419 (+ 2166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.239444 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.242346 (+ 2902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.243053 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.243223 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:02.248060 (+ 4837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:02.248069 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:02.248074 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:02.248121 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:02.252295 (+ 4174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:02.253237 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:02.253242 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:02.253247 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:02.253426 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:02.253967 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:02.253972 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:02.253976 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:02.254248 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:02.254450 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:02.258059 (+ 3609us) server_negotiation.cc:299] Negotiation successful
0924 00:56:02.258368 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260924 00:56:02.282092 10307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:02.269134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:47163 (local address 127.8.64.190:44123)
0924 00:56:02.269298 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:02.269302 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:02.269851 (+ 549us) server_negotiation.cc:407] Connection header received
0924 00:56:02.270047 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:02.270053 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:02.270134 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:02.270238 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:02.271492 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.272185 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.273460 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.273907 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.274054 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:02.276749 (+ 2695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:02.276754 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:02.276757 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:02.276785 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:02.279373 (+ 2588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:02.280085 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:02.280091 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:02.280104 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:02.280231 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:02.280584 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:02.280590 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:02.280594 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:02.280855 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:02.281016 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:02.281399 (+ 383us) server_negotiation.cc:299] Negotiation successful
0924 00:56:02.281739 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260924 00:56:02.282490 10677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:02.269134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44123 (local address 127.8.64.131:47163)
0924 00:56:02.269749 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:02.269765 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:02.269918 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:02.270258 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:02.270263 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:02.270850 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:02.271287 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:02.271349 (+ 62us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.272313 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.272320 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:02.273324 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:02.273334 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.274105 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.274111 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:02.274270 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:02.275066 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:02.275101 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:02.276546 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:02.279539 (+ 2993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:02.279544 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:02.279547 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:02.279959 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:02.280353 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:02.280362 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:02.280364 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:02.280477 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:02.280983 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:02.280987 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:02.281161 (+ 174us) client_negotiation.cc:769] Sending connection context
0924 00:56:02.281304 (+ 143us) client_negotiation.cc:240] Negotiation successful
0924 00:56:02.281741 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":358,"threads_started":1}
I20260924 00:56:02.288355 10420 heartbeater.cc:503] Master 127.8.64.190:44123 was elected leader, sending a full tablet report...
I20260924 00:56:02.292156 10563 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.131:35983
I20260924 00:56:02.292222 10697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.131:35983 every 8 connection(s)
I20260924 00:56:02.293309 10563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
I20260924 00:56:02.295298 10698 heartbeater.cc:347] Connected to a master server at 127.8.64.190:44123
I20260924 00:56:02.295526 10698 heartbeater.cc:464] Registering TS with master...
I20260924 00:56:02.296154 10698 heartbeater.cc:511] Master 127.8.64.190:44123 requested a full tablet report, sending...
I20260924 00:56:02.297410 10210 ts_manager.cc:194] Registered new tserver with Master: 08a0bd05c3b04ce594713e51cec5e82a (127.8.64.131:35983)
I20260924 00:56:02.298378 10210 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.131@KRBTEST.COM'} at 127.8.64.131:38619
I20260924 00:56:02.302088 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 10563
I20260924 00:56:02.302227 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance
I20260924 00:56:02.302695 8450 external_mini_cluster.cc:1428] Setting key c4ebe08101297dde189ae39f23dd8661
I20260924 00:56:02.305352 8450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 24 00:56:02 dist-test-slave-mmhs krb5kdc[10166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211362, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 24 00:56:02 dist-test-slave-mmhs krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211362, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:56:02.351337 10307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:02.338080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56768 (local address 127.8.64.190:44123)
0924 00:56:02.338325 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:02.338332 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:02.338518 (+ 186us) server_negotiation.cc:407] Connection header received
0924 00:56:02.338738 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:02.338743 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:02.338814 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:02.338922 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:02.340254 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.340988 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.342205 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.342692 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.342821 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:02.346476 (+ 3655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:02.346481 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:02.346484 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:02.346516 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:02.348768 (+ 2252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:02.349394 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:02.349400 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:02.349403 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:02.349535 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:02.349899 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:02.349905 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:02.349908 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:02.350183 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:02.350447 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:02.350751 (+ 304us) server_negotiation.cc:299] Negotiation successful
0924 00:56:02.350963 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260924 00:56:02.355541 10210 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56768:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:56:02.363102 10210 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:56:02.384687 10711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:02.374330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.131:35983 (local address 127.0.0.1:60822)
0924 00:56:02.375125 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:02.375165 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:02.375366 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:02.377103 (+ 1737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:02.377110 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:02.377140 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:56:02.377602 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:02.377624 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.379826 (+ 2202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.379831 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:02.381975 (+ 2144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:02.381985 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.383575 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.383581 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:02.383828 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:02.383895 (+ 67us) client_negotiation.cc:769] Sending connection context
0924 00:56:02.384018 (+ 123us) client_negotiation.cc:240] Negotiation successful
0924 00:56:02.384225 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"mutex_wait_us":21,"thread_start_us":333,"threads_started":1}
I20260924 00:56:02.385138 10712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:02.374608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60822 (local address 127.8.64.131:35983)
0924 00:56:02.376393 (+ 1785us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:02.376401 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:02.376433 (+ 32us) server_negotiation.cc:407] Connection header received
0924 00:56:02.376548 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:02.376558 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:02.376854 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:02.377865 (+ 1011us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:56:02.378230 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.379676 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.382140 (+ 2464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.383323 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.384023 (+ 700us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:02.384251 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:02.384319 (+ 68us) server_negotiation.cc:299] Negotiation successful
0924 00:56:02.384480 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1620,"thread_start_us":2642,"threads_started":1}
I20260924 00:56:02.386360 10713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:02.374619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:46381 (local address 127.0.0.1:43662)
0924 00:56:02.377967 (+ 3348us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:02.377985 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:02.378108 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:02.378971 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:02.378975 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:02.379001 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:56:02.379364 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:02.379380 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.381208 (+ 1828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.381212 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:02.383475 (+ 2263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:02.383492 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.385642 (+ 2150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.385647 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:02.385755 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:02.385776 (+ 21us) client_negotiation.cc:769] Sending connection context
0924 00:56:02.385886 (+ 110us) client_negotiation.cc:240] Negotiation successful
0924 00:56:02.386003 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3213,"thread_start_us":3046,"threads_started":1}
I20260924 00:56:02.386600 10714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:02.375075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43662 (local address 127.8.64.129:46381)
0924 00:56:02.375909 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:02.375917 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:02.378246 (+ 2329us) server_negotiation.cc:407] Connection header received
0924 00:56:02.378380 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:02.378389 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:02.378626 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:02.378811 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:56:02.379551 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.381067 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.383662 (+ 2595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.385433 (+ 1771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.385570 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:02.385750 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:02.385933 (+ 183us) server_negotiation.cc:299] Negotiation successful
0924 00:56:02.386089 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":693,"thread_start_us":587,"threads_started":1}
I20260924 00:56:02.388871 10716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:02.377079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34520 (local address 127.8.64.130:45425)
0924 00:56:02.378992 (+ 1913us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:02.378999 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:02.379023 (+ 24us) server_negotiation.cc:407] Connection header received
0924 00:56:02.379115 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:02.379123 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:02.379348 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:02.379516 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:56:02.381565 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.383045 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.386287 (+ 3242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.387594 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.387730 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:02.387902 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:02.388406 (+ 504us) server_negotiation.cc:299] Negotiation successful
0924 00:56:02.388571 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1768,"thread_start_us":1621,"threads_started":1}
I20260924 00:56:02.389048 10715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:02.376419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.130:45425 (local address 127.0.0.1:34520)
0924 00:56:02.378340 (+ 1921us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:02.378363 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:02.378503 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:02.379730 (+ 1227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:02.379734 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:02.379759 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:56:02.381353 (+ 1594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:02.381366 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.383201 (+ 1835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.383206 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:02.386097 (+ 2891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:02.386116 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.388032 (+ 1916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.388050 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:02.388213 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:02.388243 (+ 30us) client_negotiation.cc:769] Sending connection context
0924 00:56:02.388342 (+ 99us) client_negotiation.cc:240] Negotiation successful
0924 00:56:02.388699 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1763,"mutex_wait_us":6,"thread_start_us":1616,"threads_started":1}
I20260924 00:56:02.389048 10628 tablet_service.cc:1467] Processing CreateTablet for tablet e57ca99c02e041a1a96b5f6fb80e2b25 (DEFAULT_TABLE table=test-table [id=64e8f0c20d0544b78b7f7b9cd06bcba2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:56:02.389842 10349 tablet_service.cc:1467] Processing CreateTablet for tablet e57ca99c02e041a1a96b5f6fb80e2b25 (DEFAULT_TABLE table=test-table [id=64e8f0c20d0544b78b7f7b9cd06bcba2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:56:02.390599 10628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e57ca99c02e041a1a96b5f6fb80e2b25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:02.391162 10349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e57ca99c02e041a1a96b5f6fb80e2b25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:02.392448 10489 tablet_service.cc:1467] Processing CreateTablet for tablet e57ca99c02e041a1a96b5f6fb80e2b25 (DEFAULT_TABLE table=test-table [id=64e8f0c20d0544b78b7f7b9cd06bcba2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:56:02.393718 10489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e57ca99c02e041a1a96b5f6fb80e2b25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:02.401994 10717 tablet_bootstrap.cc:492] T e57ca99c02e041a1a96b5f6fb80e2b25 P 08a0bd05c3b04ce594713e51cec5e82a: Bootstrap starting.
I20260924 00:56:02.405256 10717 tablet_bootstrap.cc:654] T e57ca99c02e041a1a96b5f6fb80e2b25 P 08a0bd05c3b04ce594713e51cec5e82a: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:02.406731 10717 log.cc:824] T e57ca99c02e041a1a96b5f6fb80e2b25 P 08a0bd05c3b04ce594713e51cec5e82a: Log is configured to *not* fsync() on all Append() calls
I20260924 00:56:02.406956 10718 tablet_bootstrap.cc:492] T e57ca99c02e041a1a96b5f6fb80e2b25 P 70748f17176b4448ad9cfbe5e1e7c33e: Bootstrap starting.
I20260924 00:56:02.410386 10717 tablet_bootstrap.cc:492] T e57ca99c02e041a1a96b5f6fb80e2b25 P 08a0bd05c3b04ce594713e51cec5e82a: No bootstrap required, opened a new log
I20260924 00:56:02.410565 10720 tablet_bootstrap.cc:492] T e57ca99c02e041a1a96b5f6fb80e2b25 P e51782766430416cba83a564995c41ba: Bootstrap starting.
I20260924 00:56:02.410663 10717 ts_tablet_manager.cc:1397] T e57ca99c02e041a1a96b5f6fb80e2b25 P 08a0bd05c3b04ce594713e51cec5e82a: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260924 00:56:02.410730 10718 tablet_bootstrap.cc:654] T e57ca99c02e041a1a96b5f6fb80e2b25 P 70748f17176b4448ad9cfbe5e1e7c33e: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:02.411952 10718 log.cc:824] T e57ca99c02e041a1a96b5f6fb80e2b25 P 70748f17176b4448ad9cfbe5e1e7c33e: Log is configured to *not* fsync() on all Append() calls
I20260924 00:56:02.413532 10720 tablet_bootstrap.cc:654] T e57ca99c02e041a1a96b5f6fb80e2b25 P e51782766430416cba83a564995c41ba: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:02.414716 10720 log.cc:824] T e57ca99c02e041a1a96b5f6fb80e2b25 P e51782766430416cba83a564995c41ba: Log is configured to *not* fsync() on all Append() calls
I20260924 00:56:02.415678 10718 tablet_bootstrap.cc:492] T e57ca99c02e041a1a96b5f6fb80e2b25 P 70748f17176b4448ad9cfbe5e1e7c33e: No bootstrap required, opened a new log
I20260924 00:56:02.415940 10718 ts_tablet_manager.cc:1397] T e57ca99c02e041a1a96b5f6fb80e2b25 P 70748f17176b4448ad9cfbe5e1e7c33e: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260924 00:56:02.417214 10720 tablet_bootstrap.cc:492] T e57ca99c02e041a1a96b5f6fb80e2b25 P e51782766430416cba83a564995c41ba: No bootstrap required, opened a new log
I20260924 00:56:02.417467 10720 ts_tablet_manager.cc:1397] T e57ca99c02e041a1a96b5f6fb80e2b25 P e51782766430416cba83a564995c41ba: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260924 00:56:02.420953 10717 raft_consensus.cc:348] T e57ca99c02e041a1a96b5f6fb80e2b25 P 08a0bd05c3b04ce594713e51cec5e82a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } } peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } }
I20260924 00:56:02.421386 10717 raft_consensus.cc:374] T e57ca99c02e041a1a96b5f6fb80e2b25 P 08a0bd05c3b04ce594713e51cec5e82a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:02.421479 10717 raft_consensus.cc:729] T e57ca99c02e041a1a96b5f6fb80e2b25 P 08a0bd05c3b04ce594713e51cec5e82a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08a0bd05c3b04ce594713e51cec5e82a, State: Initialized, Role: FOLLOWER
I20260924 00:56:02.422091 10717 consensus_queue.cc:260] T e57ca99c02e041a1a96b5f6fb80e2b25 P 08a0bd05c3b04ce594713e51cec5e82a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } } peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } }
I20260924 00:56:02.423166 10698 heartbeater.cc:503] Master 127.8.64.190:44123 was elected leader, sending a full tablet report...
I20260924 00:56:02.423799 10717 ts_tablet_manager.cc:1428] T e57ca99c02e041a1a96b5f6fb80e2b25 P 08a0bd05c3b04ce594713e51cec5e82a: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260924 00:56:02.426399 10718 raft_consensus.cc:348] T e57ca99c02e041a1a96b5f6fb80e2b25 P 70748f17176b4448ad9cfbe5e1e7c33e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } } peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } }
I20260924 00:56:02.426831 10718 raft_consensus.cc:374] T e57ca99c02e041a1a96b5f6fb80e2b25 P 70748f17176b4448ad9cfbe5e1e7c33e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:02.426939 10718 raft_consensus.cc:729] T e57ca99c02e041a1a96b5f6fb80e2b25 P 70748f17176b4448ad9cfbe5e1e7c33e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70748f17176b4448ad9cfbe5e1e7c33e, State: Initialized, Role: FOLLOWER
I20260924 00:56:02.427016 10720 raft_consensus.cc:348] T e57ca99c02e041a1a96b5f6fb80e2b25 P e51782766430416cba83a564995c41ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } } peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } }
I20260924 00:56:02.427428 10720 raft_consensus.cc:374] T e57ca99c02e041a1a96b5f6fb80e2b25 P e51782766430416cba83a564995c41ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:02.427542 10720 raft_consensus.cc:729] T e57ca99c02e041a1a96b5f6fb80e2b25 P e51782766430416cba83a564995c41ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51782766430416cba83a564995c41ba, State: Initialized, Role: FOLLOWER
I20260924 00:56:02.427523 10718 consensus_queue.cc:260] T e57ca99c02e041a1a96b5f6fb80e2b25 P 70748f17176b4448ad9cfbe5e1e7c33e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } } peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } }
I20260924 00:56:02.428249 10720 consensus_queue.cc:260] T e57ca99c02e041a1a96b5f6fb80e2b25 P e51782766430416cba83a564995c41ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } } peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } }
I20260924 00:56:02.429411 10559 heartbeater.cc:503] Master 127.8.64.190:44123 was elected leader, sending a full tablet report...
I20260924 00:56:02.429965 10720 ts_tablet_manager.cc:1428] T e57ca99c02e041a1a96b5f6fb80e2b25 P e51782766430416cba83a564995c41ba: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260924 00:56:02.431658 10718 ts_tablet_manager.cc:1428] T e57ca99c02e041a1a96b5f6fb80e2b25 P 70748f17176b4448ad9cfbe5e1e7c33e: Time spent starting tablet: real 0.016s user 0.010s sys 0.005s
I20260924 00:56:02.465387 10725 raft_consensus.cc:482] T e57ca99c02e041a1a96b5f6fb80e2b25 P e51782766430416cba83a564995c41ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:56:02.465731 10725 raft_consensus.cc:504] T e57ca99c02e041a1a96b5f6fb80e2b25 P e51782766430416cba83a564995c41ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } } peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } }
I20260924 00:56:02.467533 10725 leader_election.cc:288] T e57ca99c02e041a1a96b5f6fb80e2b25 P e51782766430416cba83a564995c41ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08a0bd05c3b04ce594713e51cec5e82a (127.8.64.131:35983), 70748f17176b4448ad9cfbe5e1e7c33e (127.8.64.129:46381)
I20260924 00:56:02.474577 10714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:02.468128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:35833 (local address 127.8.64.129:46381)
0924 00:56:02.468359 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:02.468368 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:02.468961 (+ 593us) server_negotiation.cc:407] Connection header received
0924 00:56:02.469186 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:02.469193 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:02.469267 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:02.469368 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:56:02.469920 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.470700 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.472442 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.473422 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.473594 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:02.473673 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:02.474165 (+ 492us) server_negotiation.cc:299] Negotiation successful
0924 00:56:02.474317 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260924 00:56:02.474581 10712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:02.467958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:34497 (local address 127.8.64.131:35983)
0924 00:56:02.468186 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:02.468196 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:02.469191 (+ 995us) server_negotiation.cc:407] Connection header received
0924 00:56:02.469382 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:02.469389 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:02.469450 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:02.469545 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:56:02.470165 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.470915 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.472605 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.473632 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.473798 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:02.473864 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:02.474159 (+ 295us) server_negotiation.cc:299] Negotiation successful
0924 00:56:02.474318 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260924 00:56:02.474784 10726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:02.467866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:46381 (local address 127.8.64.130:35833)
0924 00:56:02.468845 (+ 979us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:02.468864 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:02.469038 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:02.469386 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:02.469390 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:02.469414 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:56:02.469780 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:02.469789 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.470833 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.470841 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:02.472287 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:02.472298 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.473631 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.473776 (+ 145us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:02.474006 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:02.474030 (+ 24us) client_negotiation.cc:769] Sending connection context
0924 00:56:02.474139 (+ 109us) client_negotiation.cc:240] Negotiation successful
0924 00:56:02.474325 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":874,"mutex_wait_us":38,"thread_start_us":502,"threads_started":1}
I20260924 00:56:02.474825 10727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:02.467875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.131:35983 (local address 127.8.64.130:34497)
0924 00:56:02.469079 (+ 1204us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:02.469093 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:02.469239 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:02.469560 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:02.469564 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:02.469590 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:56:02.469992 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:02.470000 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.471049 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.471054 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:02.472443 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:02.472454 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.473815 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.473820 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:02.474001 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:02.474024 (+ 23us) client_negotiation.cc:769] Sending connection context
0924 00:56:02.474133 (+ 109us) client_negotiation.cc:240] Negotiation successful
0924 00:56:02.474319 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1098,"mutex_wait_us":129,"thread_start_us":771,"threads_started":1}
I20260924 00:56:02.475903 10370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e57ca99c02e041a1a96b5f6fb80e2b25" candidate_uuid: "e51782766430416cba83a564995c41ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" is_pre_election: true
I20260924 00:56:02.475980 10648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e57ca99c02e041a1a96b5f6fb80e2b25" candidate_uuid: "e51782766430416cba83a564995c41ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08a0bd05c3b04ce594713e51cec5e82a" is_pre_election: true
I20260924 00:56:02.476418 10370 raft_consensus.cc:2454] T e57ca99c02e041a1a96b5f6fb80e2b25 P 70748f17176b4448ad9cfbe5e1e7c33e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51782766430416cba83a564995c41ba in term 0.
I20260924 00:56:02.476447 10648 raft_consensus.cc:2454] T e57ca99c02e041a1a96b5f6fb80e2b25 P 08a0bd05c3b04ce594713e51cec5e82a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51782766430416cba83a564995c41ba in term 0.
I20260924 00:56:02.477082 10442 leader_election.cc:302] T e57ca99c02e041a1a96b5f6fb80e2b25 P e51782766430416cba83a564995c41ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70748f17176b4448ad9cfbe5e1e7c33e, e51782766430416cba83a564995c41ba; no voters:
I20260924 00:56:02.477480 10725 raft_consensus.cc:2781] T e57ca99c02e041a1a96b5f6fb80e2b25 P e51782766430416cba83a564995c41ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:56:02.477620 10725 raft_consensus.cc:482] T e57ca99c02e041a1a96b5f6fb80e2b25 P e51782766430416cba83a564995c41ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:56:02.477725 10725 raft_consensus.cc:3037] T e57ca99c02e041a1a96b5f6fb80e2b25 P e51782766430416cba83a564995c41ba [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:02.480036 10725 raft_consensus.cc:504] T e57ca99c02e041a1a96b5f6fb80e2b25 P e51782766430416cba83a564995c41ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } } peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } }
I20260924 00:56:02.480670 10725 leader_election.cc:288] T e57ca99c02e041a1a96b5f6fb80e2b25 P e51782766430416cba83a564995c41ba [CANDIDATE]: Term 1 election: Requested vote from peers 08a0bd05c3b04ce594713e51cec5e82a (127.8.64.131:35983), 70748f17176b4448ad9cfbe5e1e7c33e (127.8.64.129:46381)
I20260924 00:56:02.481055 10648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e57ca99c02e041a1a96b5f6fb80e2b25" candidate_uuid: "e51782766430416cba83a564995c41ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08a0bd05c3b04ce594713e51cec5e82a"
I20260924 00:56:02.481194 10370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e57ca99c02e041a1a96b5f6fb80e2b25" candidate_uuid: "e51782766430416cba83a564995c41ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70748f17176b4448ad9cfbe5e1e7c33e"
I20260924 00:56:02.481261 10648 raft_consensus.cc:3037] T e57ca99c02e041a1a96b5f6fb80e2b25 P 08a0bd05c3b04ce594713e51cec5e82a [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:02.481422 10370 raft_consensus.cc:3037] T e57ca99c02e041a1a96b5f6fb80e2b25 P 70748f17176b4448ad9cfbe5e1e7c33e [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:02.483566 10648 raft_consensus.cc:2454] T e57ca99c02e041a1a96b5f6fb80e2b25 P 08a0bd05c3b04ce594713e51cec5e82a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51782766430416cba83a564995c41ba in term 1.
I20260924 00:56:02.483670 10370 raft_consensus.cc:2454] T e57ca99c02e041a1a96b5f6fb80e2b25 P 70748f17176b4448ad9cfbe5e1e7c33e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51782766430416cba83a564995c41ba in term 1.
I20260924 00:56:02.484082 10440 leader_election.cc:302] T e57ca99c02e041a1a96b5f6fb80e2b25 P e51782766430416cba83a564995c41ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08a0bd05c3b04ce594713e51cec5e82a, e51782766430416cba83a564995c41ba; no voters:
I20260924 00:56:02.484372 10725 raft_consensus.cc:2781] T e57ca99c02e041a1a96b5f6fb80e2b25 P e51782766430416cba83a564995c41ba [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:56:02.484802 10725 raft_consensus.cc:686] T e57ca99c02e041a1a96b5f6fb80e2b25 P e51782766430416cba83a564995c41ba [term 1 LEADER]: Becoming Leader. State: Replica: e51782766430416cba83a564995c41ba, State: Running, Role: LEADER
I20260924 00:56:02.485343 10725 consensus_queue.cc:237] T e57ca99c02e041a1a96b5f6fb80e2b25 P e51782766430416cba83a564995c41ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } } peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } }
I20260924 00:56:02.489707 10210 catalog_manager.cc:5495] T e57ca99c02e041a1a96b5f6fb80e2b25 P e51782766430416cba83a564995c41ba reported cstate change: term changed from 0 to 1, leader changed from <none> to e51782766430416cba83a564995c41ba (127.8.64.130). New cstate: current_term: 1 leader_uuid: "e51782766430416cba83a564995c41ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } health_report { overall_health: HEALTHY } } }
Sep 24 00:56:02 dist-test-slave-mmhs krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211360, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
W20260924 00:56:02.531998 10421 tablet.cc:2369] T e57ca99c02e041a1a96b5f6fb80e2b25 P 70748f17176b4448ad9cfbe5e1e7c33e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:56:02.534085 10735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:02.520927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44123 (local address 127.0.0.1:56780)
0924 00:56:02.521516 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:02.521532 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:02.521669 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:02.521983 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:02.521986 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:02.522728 (+ 742us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:02.523001 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:02.523011 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.524135 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.524138 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:02.525094 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:02.525106 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.525856 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.525860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:02.525990 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:02.526635 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:02.526651 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:02.528797 (+ 2146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:02.531514 (+ 2717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:02.531523 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:02.531529 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:02.531934 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:02.532348 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:02.532354 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:02.532358 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:02.532488 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:02.532992 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:02.533000 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:02.533162 (+ 162us) client_negotiation.cc:769] Sending connection context
0924 00:56:02.533322 (+ 160us) client_negotiation.cc:240] Negotiation successful
0924 00:56:02.533596 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":335,"threads_started":1}
I20260924 00:56:02.534085 10307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:02.520862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56780 (local address 127.8.64.190:44123)
0924 00:56:02.521076 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:02.521084 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:02.521634 (+ 550us) server_negotiation.cc:407] Connection header received
0924 00:56:02.521785 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:02.521789 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:02.521848 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:02.521943 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:02.523176 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.524018 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.525236 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.525700 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.525803 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:02.528946 (+ 3143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:02.528951 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:02.528954 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:02.528982 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:02.531394 (+ 2412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:02.532070 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:02.532077 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:02.532080 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:02.532255 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:02.532593 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:02.532599 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:02.532602 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:02.532854 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:02.533020 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:02.533402 (+ 382us) server_negotiation.cc:299] Negotiation successful
0924 00:56:02.533723 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":132,"server-negotiator.queue_time_us":82}
I20260924 00:56:02.539132 10210 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:56780:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260924 00:56:02.540138 10210 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260924 00:56:02.541657 10560 tablet.cc:2369] T e57ca99c02e041a1a96b5f6fb80e2b25 P e51782766430416cba83a564995c41ba: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 00:56:02.544878 10699 tablet.cc:2369] T e57ca99c02e041a1a96b5f6fb80e2b25 P 08a0bd05c3b04ce594713e51cec5e82a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:56:02.546727 10489 tablet_service.cc:1467] Processing CreateTablet for tablet 9dcebdb4861b4852a6d32102f063e9a7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=74c928b9e5124bedb989a7688fa86dfd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 00:56:02.546653 10349 tablet_service.cc:1467] Processing CreateTablet for tablet 9dcebdb4861b4852a6d32102f063e9a7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=74c928b9e5124bedb989a7688fa86dfd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 00:56:02.546924 10628 tablet_service.cc:1467] Processing CreateTablet for tablet 9dcebdb4861b4852a6d32102f063e9a7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=74c928b9e5124bedb989a7688fa86dfd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 00:56:02.547593 10489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dcebdb4861b4852a6d32102f063e9a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:02.547626 10628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dcebdb4861b4852a6d32102f063e9a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:02.547626 10349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dcebdb4861b4852a6d32102f063e9a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:02.553223 10720 tablet_bootstrap.cc:492] T 9dcebdb4861b4852a6d32102f063e9a7 P e51782766430416cba83a564995c41ba: Bootstrap starting.
I20260924 00:56:02.554507 10718 tablet_bootstrap.cc:492] T 9dcebdb4861b4852a6d32102f063e9a7 P 70748f17176b4448ad9cfbe5e1e7c33e: Bootstrap starting.
I20260924 00:56:02.554661 10720 tablet_bootstrap.cc:654] T 9dcebdb4861b4852a6d32102f063e9a7 P e51782766430416cba83a564995c41ba: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:02.556046 10718 tablet_bootstrap.cc:654] T 9dcebdb4861b4852a6d32102f063e9a7 P 70748f17176b4448ad9cfbe5e1e7c33e: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:02.557296 10717 tablet_bootstrap.cc:492] T 9dcebdb4861b4852a6d32102f063e9a7 P 08a0bd05c3b04ce594713e51cec5e82a: Bootstrap starting.
I20260924 00:56:02.558065 10718 tablet_bootstrap.cc:492] T 9dcebdb4861b4852a6d32102f063e9a7 P 70748f17176b4448ad9cfbe5e1e7c33e: No bootstrap required, opened a new log
I20260924 00:56:02.558226 10718 ts_tablet_manager.cc:1397] T 9dcebdb4861b4852a6d32102f063e9a7 P 70748f17176b4448ad9cfbe5e1e7c33e: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260924 00:56:02.558951 10718 raft_consensus.cc:348] T 9dcebdb4861b4852a6d32102f063e9a7 P 70748f17176b4448ad9cfbe5e1e7c33e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } }
I20260924 00:56:02.559167 10718 raft_consensus.cc:374] T 9dcebdb4861b4852a6d32102f063e9a7 P 70748f17176b4448ad9cfbe5e1e7c33e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:02.559265 10718 raft_consensus.cc:729] T 9dcebdb4861b4852a6d32102f063e9a7 P 70748f17176b4448ad9cfbe5e1e7c33e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70748f17176b4448ad9cfbe5e1e7c33e, State: Initialized, Role: FOLLOWER
I20260924 00:56:02.559491 10718 consensus_queue.cc:260] T 9dcebdb4861b4852a6d32102f063e9a7 P 70748f17176b4448ad9cfbe5e1e7c33e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } }
I20260924 00:56:02.559628 10717 tablet_bootstrap.cc:654] T 9dcebdb4861b4852a6d32102f063e9a7 P 08a0bd05c3b04ce594713e51cec5e82a: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:02.560068 10718 ts_tablet_manager.cc:1428] T 9dcebdb4861b4852a6d32102f063e9a7 P 70748f17176b4448ad9cfbe5e1e7c33e: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260924 00:56:02.560417 10724 tablet_replica.cc:439] T 9dcebdb4861b4852a6d32102f063e9a7 P 70748f17176b4448ad9cfbe5e1e7c33e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } } }
I20260924 00:56:02.560717 10724 tablet_replica.cc:442] T 9dcebdb4861b4852a6d32102f063e9a7 P 70748f17176b4448ad9cfbe5e1e7c33e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 00:56:02.562436 10717 tablet_bootstrap.cc:492] T 9dcebdb4861b4852a6d32102f063e9a7 P 08a0bd05c3b04ce594713e51cec5e82a: No bootstrap required, opened a new log
I20260924 00:56:02.562582 10717 ts_tablet_manager.cc:1397] T 9dcebdb4861b4852a6d32102f063e9a7 P 08a0bd05c3b04ce594713e51cec5e82a: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.004s
I20260924 00:56:02.562873 10720 tablet_bootstrap.cc:492] T 9dcebdb4861b4852a6d32102f063e9a7 P e51782766430416cba83a564995c41ba: No bootstrap required, opened a new log
I20260924 00:56:02.563014 10720 ts_tablet_manager.cc:1397] T 9dcebdb4861b4852a6d32102f063e9a7 P e51782766430416cba83a564995c41ba: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260924 00:56:02.563438 10717 raft_consensus.cc:348] T 9dcebdb4861b4852a6d32102f063e9a7 P 08a0bd05c3b04ce594713e51cec5e82a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } }
I20260924 00:56:02.563683 10717 raft_consensus.cc:374] T 9dcebdb4861b4852a6d32102f063e9a7 P 08a0bd05c3b04ce594713e51cec5e82a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:02.563787 10717 raft_consensus.cc:729] T 9dcebdb4861b4852a6d32102f063e9a7 P 08a0bd05c3b04ce594713e51cec5e82a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08a0bd05c3b04ce594713e51cec5e82a, State: Initialized, Role: FOLLOWER
I20260924 00:56:02.563808 10720 raft_consensus.cc:348] T 9dcebdb4861b4852a6d32102f063e9a7 P e51782766430416cba83a564995c41ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } }
I20260924 00:56:02.564028 10720 raft_consensus.cc:374] T 9dcebdb4861b4852a6d32102f063e9a7 P e51782766430416cba83a564995c41ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:02.564131 10720 raft_consensus.cc:729] T 9dcebdb4861b4852a6d32102f063e9a7 P e51782766430416cba83a564995c41ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51782766430416cba83a564995c41ba, State: Initialized, Role: FOLLOWER
I20260924 00:56:02.564020 10717 consensus_queue.cc:260] T 9dcebdb4861b4852a6d32102f063e9a7 P 08a0bd05c3b04ce594713e51cec5e82a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } }
I20260924 00:56:02.564374 10720 consensus_queue.cc:260] T 9dcebdb4861b4852a6d32102f063e9a7 P e51782766430416cba83a564995c41ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } }
I20260924 00:56:02.564477 10723 tablet_replica.cc:439] T 9dcebdb4861b4852a6d32102f063e9a7 P 08a0bd05c3b04ce594713e51cec5e82a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } } }
I20260924 00:56:02.564805 10723 tablet_replica.cc:442] T 9dcebdb4861b4852a6d32102f063e9a7 P 08a0bd05c3b04ce594713e51cec5e82a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 00:56:02.564776 10725 tablet_replica.cc:439] T 9dcebdb4861b4852a6d32102f063e9a7 P e51782766430416cba83a564995c41ba: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } } }
I20260924 00:56:02.565014 10725 tablet_replica.cc:442] T 9dcebdb4861b4852a6d32102f063e9a7 P e51782766430416cba83a564995c41ba: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 00:56:02.565725 10717 ts_tablet_manager.cc:1428] T 9dcebdb4861b4852a6d32102f063e9a7 P 08a0bd05c3b04ce594713e51cec5e82a: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260924 00:56:02.566529 10720 ts_tablet_manager.cc:1428] T 9dcebdb4861b4852a6d32102f063e9a7 P e51782766430416cba83a564995c41ba: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260924 00:56:02.770079 10724 raft_consensus.cc:482] T 9dcebdb4861b4852a6d32102f063e9a7 P 70748f17176b4448ad9cfbe5e1e7c33e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:56:02.770399 10724 raft_consensus.cc:504] T 9dcebdb4861b4852a6d32102f063e9a7 P 70748f17176b4448ad9cfbe5e1e7c33e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } }
I20260924 00:56:02.772078 10724 leader_election.cc:288] T 9dcebdb4861b4852a6d32102f063e9a7 P 70748f17176b4448ad9cfbe5e1e7c33e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e51782766430416cba83a564995c41ba (127.8.64.130:45425), 08a0bd05c3b04ce594713e51cec5e82a (127.8.64.131:35983)
I20260924 00:56:02.778896 10716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:02.772443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:39605 (local address 127.8.64.130:45425)
0924 00:56:02.772681 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:02.772688 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:02.773432 (+ 744us) server_negotiation.cc:407] Connection header received
0924 00:56:02.773668 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:02.773673 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:02.773730 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:02.773828 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:56:02.774477 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.775189 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.777140 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.777898 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.778089 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:02.778172 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:02.778500 (+ 328us) server_negotiation.cc:299] Negotiation successful
0924 00:56:02.778622 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260924 00:56:02.778978 10712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:02.772526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:60759 (local address 127.8.64.131:35983)
0924 00:56:02.772789 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:02.772797 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:02.773529 (+ 732us) server_negotiation.cc:407] Connection header received
0924 00:56:02.773723 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:02.773728 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:02.773785 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:02.773879 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:56:02.774495 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.775192 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.777082 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.777996 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.778182 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:02.778257 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:02.778592 (+ 335us) server_negotiation.cc:299] Negotiation successful
0924 00:56:02.778714 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260924 00:56:02.779337 10740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:02.772328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.130:45425 (local address 127.8.64.129:39605)
0924 00:56:02.773316 (+ 988us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:02.773332 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:02.773521 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:02.773846 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:02.773850 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:02.773874 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:56:02.774273 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:02.774285 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.775319 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.775327 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:02.776977 (+ 1650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:02.776989 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.778099 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.778105 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:02.778275 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:02.778366 (+ 91us) client_negotiation.cc:769] Sending connection context
0924 00:56:02.778476 (+ 110us) client_negotiation.cc:240] Negotiation successful
0924 00:56:02.778615 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":848,"mutex_wait_us":57,"thread_start_us":764,"threads_started":1}
I20260924 00:56:02.779378 10739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:02.772594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.131:35983 (local address 127.8.64.129:60759)
0924 00:56:02.773414 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:02.773428 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:02.773585 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:02.773903 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:02.773907 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:02.773934 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:56:02.774336 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:02.774354 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.775328 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.775333 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:02.776928 (+ 1595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:02.776940 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.778219 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.778226 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:02.778437 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:02.778459 (+ 22us) client_negotiation.cc:769] Sending connection context
0924 00:56:02.778566 (+ 107us) client_negotiation.cc:240] Negotiation successful
0924 00:56:02.778688 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"mutex_wait_us":52,"thread_start_us":406,"threads_started":1}
I20260924 00:56:02.780076 10648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dcebdb4861b4852a6d32102f063e9a7" candidate_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08a0bd05c3b04ce594713e51cec5e82a" is_pre_election: true
I20260924 00:56:02.780345 10648 raft_consensus.cc:2454] T 9dcebdb4861b4852a6d32102f063e9a7 P 08a0bd05c3b04ce594713e51cec5e82a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70748f17176b4448ad9cfbe5e1e7c33e in term 0.
I20260924 00:56:02.780282 10514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dcebdb4861b4852a6d32102f063e9a7" candidate_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51782766430416cba83a564995c41ba" is_pre_election: true
I20260924 00:56:02.780696 10514 raft_consensus.cc:2454] T 9dcebdb4861b4852a6d32102f063e9a7 P e51782766430416cba83a564995c41ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70748f17176b4448ad9cfbe5e1e7c33e in term 0.
I20260924 00:56:02.780855 10299 leader_election.cc:302] T 9dcebdb4861b4852a6d32102f063e9a7 P 70748f17176b4448ad9cfbe5e1e7c33e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08a0bd05c3b04ce594713e51cec5e82a, 70748f17176b4448ad9cfbe5e1e7c33e; no voters:
I20260924 00:56:02.781286 10724 raft_consensus.cc:2781] T 9dcebdb4861b4852a6d32102f063e9a7 P 70748f17176b4448ad9cfbe5e1e7c33e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:56:02.781418 10724 raft_consensus.cc:482] T 9dcebdb4861b4852a6d32102f063e9a7 P 70748f17176b4448ad9cfbe5e1e7c33e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:56:02.781519 10724 raft_consensus.cc:3037] T 9dcebdb4861b4852a6d32102f063e9a7 P 70748f17176b4448ad9cfbe5e1e7c33e [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:02.783946 10724 raft_consensus.cc:504] T 9dcebdb4861b4852a6d32102f063e9a7 P 70748f17176b4448ad9cfbe5e1e7c33e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } }
I20260924 00:56:02.784488 10724 leader_election.cc:288] T 9dcebdb4861b4852a6d32102f063e9a7 P 70748f17176b4448ad9cfbe5e1e7c33e [CANDIDATE]: Term 1 election: Requested vote from peers e51782766430416cba83a564995c41ba (127.8.64.130:45425), 08a0bd05c3b04ce594713e51cec5e82a (127.8.64.131:35983)
I20260924 00:56:02.784931 10648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dcebdb4861b4852a6d32102f063e9a7" candidate_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08a0bd05c3b04ce594713e51cec5e82a"
I20260924 00:56:02.785123 10648 raft_consensus.cc:3037] T 9dcebdb4861b4852a6d32102f063e9a7 P 08a0bd05c3b04ce594713e51cec5e82a [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:02.785142 10514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dcebdb4861b4852a6d32102f063e9a7" candidate_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51782766430416cba83a564995c41ba"
I20260924 00:56:02.785362 10514 raft_consensus.cc:3037] T 9dcebdb4861b4852a6d32102f063e9a7 P e51782766430416cba83a564995c41ba [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:02.787359 10648 raft_consensus.cc:2454] T 9dcebdb4861b4852a6d32102f063e9a7 P 08a0bd05c3b04ce594713e51cec5e82a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70748f17176b4448ad9cfbe5e1e7c33e in term 1.
I20260924 00:56:02.787595 10514 raft_consensus.cc:2454] T 9dcebdb4861b4852a6d32102f063e9a7 P e51782766430416cba83a564995c41ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70748f17176b4448ad9cfbe5e1e7c33e in term 1.
I20260924 00:56:02.787788 10299 leader_election.cc:302] T 9dcebdb4861b4852a6d32102f063e9a7 P 70748f17176b4448ad9cfbe5e1e7c33e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08a0bd05c3b04ce594713e51cec5e82a, 70748f17176b4448ad9cfbe5e1e7c33e; no voters:
I20260924 00:56:02.788085 10724 raft_consensus.cc:2781] T 9dcebdb4861b4852a6d32102f063e9a7 P 70748f17176b4448ad9cfbe5e1e7c33e [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:56:02.788511 10724 raft_consensus.cc:686] T 9dcebdb4861b4852a6d32102f063e9a7 P 70748f17176b4448ad9cfbe5e1e7c33e [term 1 LEADER]: Becoming Leader. State: Replica: 70748f17176b4448ad9cfbe5e1e7c33e, State: Running, Role: LEADER
I20260924 00:56:02.789183 10724 consensus_queue.cc:237] T 9dcebdb4861b4852a6d32102f063e9a7 P 70748f17176b4448ad9cfbe5e1e7c33e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } }
I20260924 00:56:02.792610 10741 tablet_replica.cc:439] T 9dcebdb4861b4852a6d32102f063e9a7 P 70748f17176b4448ad9cfbe5e1e7c33e: TxnStatusTablet state changed. Reason: New leader 70748f17176b4448ad9cfbe5e1e7c33e. Latest consensus state: current_term: 1 leader_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } } }
I20260924 00:56:02.793030 10741 tablet_replica.cc:442] T 9dcebdb4861b4852a6d32102f063e9a7 P 70748f17176b4448ad9cfbe5e1e7c33e: This TxnStatusTablet replica's current role is: LEADER
I20260924 00:56:02.794716 10210 catalog_manager.cc:5495] T 9dcebdb4861b4852a6d32102f063e9a7 P 70748f17176b4448ad9cfbe5e1e7c33e reported cstate change: term changed from 0 to 1, leader changed from <none> to 70748f17176b4448ad9cfbe5e1e7c33e (127.8.64.129). New cstate: current_term: 1 leader_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } health_report { overall_health: UNKNOWN } } }
I20260924 00:56:02.802063 10743 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260924 00:56:02.802414 10743 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260924 00:56:02.852473 10735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:02.846885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:46381 (local address 127.0.0.1:43672)
0924 00:56:02.847117 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:02.847141 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:02.847279 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:02.847795 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:02.847800 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:02.847813 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 00:56:02.847997 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:02.848005 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.849028 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.849037 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:02.850014 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:02.850025 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.850821 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.850827 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:02.851006 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:02.851059 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 00:56:02.851927 (+ 868us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 00:56:02.851946 (+ 19us) client_negotiation.cc:769] Sending connection context
0924 00:56:02.852036 (+ 90us) client_negotiation.cc:240] Negotiation successful
0924 00:56:02.852179 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260924 00:56:02.852512 10714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:02.847065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43672 (local address 127.8.64.129:46381)
0924 00:56:02.847252 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:02.847258 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:02.847277 (+ 19us) server_negotiation.cc:407] Connection header received
0924 00:56:02.847508 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:02.847512 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:02.847594 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:02.847761 (+ 167us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:56:02.848133 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.848888 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.850160 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.850658 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.850806 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:02.851260 (+ 454us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:56:02.851776 (+ 516us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:56:02.851961 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:02.852076 (+ 115us) server_negotiation.cc:299] Negotiation successful
0924 00:56:02.852154 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260924 00:56:02.860165 10648 raft_consensus.cc:1261] T 9dcebdb4861b4852a6d32102f063e9a7 P 08a0bd05c3b04ce594713e51cec5e82a [term 1 FOLLOWER]: Refusing update from remote peer 70748f17176b4448ad9cfbe5e1e7c33e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:56:02.860157 10514 raft_consensus.cc:1261] T 9dcebdb4861b4852a6d32102f063e9a7 P e51782766430416cba83a564995c41ba [term 1 FOLLOWER]: Refusing update from remote peer 70748f17176b4448ad9cfbe5e1e7c33e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:56:02.861238 10741 consensus_queue.cc:1035] T 9dcebdb4861b4852a6d32102f063e9a7 P 70748f17176b4448ad9cfbe5e1e7c33e [LEADER]: Connected to new peer: Peer: permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:56:02.861608 10724 consensus_queue.cc:1035] T 9dcebdb4861b4852a6d32102f063e9a7 P 70748f17176b4448ad9cfbe5e1e7c33e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:56:02.869561 10723 tablet_replica.cc:439] T 9dcebdb4861b4852a6d32102f063e9a7 P 08a0bd05c3b04ce594713e51cec5e82a: TxnStatusTablet state changed. Reason: New leader 70748f17176b4448ad9cfbe5e1e7c33e. Latest consensus state: current_term: 1 leader_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } } }
I20260924 00:56:02.869839 10723 tablet_replica.cc:442] T 9dcebdb4861b4852a6d32102f063e9a7 P 08a0bd05c3b04ce594713e51cec5e82a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 00:56:02.872735 10746 mvcc.cc:205] Tried to move back new op lower bound from 7332705742265421824 to 7332705741988872192. Current Snapshot: MvccSnapshot[applied={T|T < 7332705742265421824}]
I20260924 00:56:02.873519 10741 tablet_replica.cc:439] T 9dcebdb4861b4852a6d32102f063e9a7 P 70748f17176b4448ad9cfbe5e1e7c33e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } } }
I20260924 00:56:02.873786 10741 tablet_replica.cc:442] T 9dcebdb4861b4852a6d32102f063e9a7 P 70748f17176b4448ad9cfbe5e1e7c33e: This TxnStatusTablet replica's current role is: LEADER
I20260924 00:56:02.874768 10748 mvcc.cc:205] Tried to move back new op lower bound from 7332705742265421824 to 7332705741988872192. Current Snapshot: MvccSnapshot[applied={T|T < 7332705742265421824}]
I20260924 00:56:02.879667 10723 tablet_replica.cc:439] T 9dcebdb4861b4852a6d32102f063e9a7 P 08a0bd05c3b04ce594713e51cec5e82a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } } }
I20260924 00:56:02.879904 10723 tablet_replica.cc:442] T 9dcebdb4861b4852a6d32102f063e9a7 P 08a0bd05c3b04ce594713e51cec5e82a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 00:56:02.887673 10725 tablet_replica.cc:439] T 9dcebdb4861b4852a6d32102f063e9a7 P e51782766430416cba83a564995c41ba: TxnStatusTablet state changed. Reason: New leader 70748f17176b4448ad9cfbe5e1e7c33e. Latest consensus state: current_term: 1 leader_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } } }
I20260924 00:56:02.888047 10725 tablet_replica.cc:442] T 9dcebdb4861b4852a6d32102f063e9a7 P e51782766430416cba83a564995c41ba: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 00:56:02.889899 10724 tablet_replica.cc:439] T 9dcebdb4861b4852a6d32102f063e9a7 P 70748f17176b4448ad9cfbe5e1e7c33e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } } }
I20260924 00:56:02.890205 10724 tablet_replica.cc:442] T 9dcebdb4861b4852a6d32102f063e9a7 P 70748f17176b4448ad9cfbe5e1e7c33e: This TxnStatusTablet replica's current role is: LEADER
I20260924 00:56:02.892176 10747 mvcc.cc:205] Tried to move back new op lower bound from 7332705742265421824 to 7332705741988872192. Current Snapshot: MvccSnapshot[applied={T|T < 7332705742265421824}]
I20260924 00:56:02.893714 10725 tablet_replica.cc:439] T 9dcebdb4861b4852a6d32102f063e9a7 P e51782766430416cba83a564995c41ba: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } } }
I20260924 00:56:02.893985 10725 tablet_replica.cc:442] T 9dcebdb4861b4852a6d32102f063e9a7 P e51782766430416cba83a564995c41ba: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 00:56:02.894845 10716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:02.883527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34530 (local address 127.8.64.130:45425)
0924 00:56:02.883756 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:02.883765 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:02.887035 (+ 3270us) server_negotiation.cc:407] Connection header received
0924 00:56:02.887366 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:02.887372 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:02.887511 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:02.887641 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:56:02.888345 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.889540 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.891794 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.892448 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.892543 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:02.893272 (+ 729us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:56:02.893986 (+ 714us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:56:02.894097 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:02.894438 (+ 341us) server_negotiation.cc:299] Negotiation successful
0924 00:56:02.894526 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260924 00:56:02.899588 10725 consensus_queue.cc:1035] T e57ca99c02e041a1a96b5f6fb80e2b25 P e51782766430416cba83a564995c41ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:56:02.908501 10725 consensus_queue.cc:1035] T e57ca99c02e041a1a96b5f6fb80e2b25 P e51782766430416cba83a564995c41ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:56:02.921231 10714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:02.906479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:36229 (local address 127.8.64.129:46381)
0924 00:56:02.906637 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:02.906643 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:02.914972 (+ 8329us) server_negotiation.cc:407] Connection header received
0924 00:56:02.915181 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:02.915186 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:02.915263 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:02.915343 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:56:02.916259 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.917442 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.919525 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.919935 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.920049 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:02.920608 (+ 559us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:56:02.920716 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:56:02.920796 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:02.920948 (+ 152us) server_negotiation.cc:299] Negotiation successful
0924 00:56:02.921013 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260924 00:56:02.921458 10759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:02.906127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:46381 (local address 127.8.64.130:36229)
0924 00:56:02.914859 (+ 8732us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:02.914877 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:02.915049 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:02.915504 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:02.915509 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:02.915522 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 00:56:02.915890 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:02.915902 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.917584 (+ 1682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.917589 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:02.919353 (+ 1764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:02.919368 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.920124 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.920131 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:02.920305 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:02.920378 (+ 73us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 00:56:02.920825 (+ 447us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 00:56:02.920836 (+ 11us) client_negotiation.cc:769] Sending connection context
0924 00:56:02.920923 (+ 87us) client_negotiation.cc:240] Negotiation successful
0924 00:56:02.921077 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8584,"thread_start_us":8392,"threads_started":1}
I20260924 00:56:02.960115 10716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:02.953839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:52733 (local address 127.8.64.130:45425)
0924 00:56:02.954038 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:02.954044 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:02.954707 (+ 663us) server_negotiation.cc:407] Connection header received
0924 00:56:02.954889 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:02.954892 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:02.954962 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:02.955034 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:56:02.955645 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.956560 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.958268 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.958801 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.958926 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:02.959388 (+ 462us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:56:02.959488 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:56:02.959589 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:02.959756 (+ 167us) server_negotiation.cc:299] Negotiation successful
0924 00:56:02.959827 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260924 00:56:02.960606 10762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:02.953797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.130:45425 (local address 127.8.64.129:52733)
0924 00:56:02.954601 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:02.954621 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:02.954773 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:02.955072 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:02.955077 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:02.955091 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 00:56:02.955491 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:02.955503 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.956688 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.956693 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:02.958129 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:02.958141 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:02.958993 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:02.958999 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:02.959151 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:02.959204 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 00:56:02.959638 (+ 434us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 00:56:02.959651 (+ 13us) client_negotiation.cc:769] Sending connection context
0924 00:56:02.959736 (+ 85us) client_negotiation.cc:240] Negotiation successful
0924 00:56:02.959863 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":363,"threads_started":1}
I20260924 00:56:03.001894 10210 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:56:03.002240 10210 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:56:03.006198 10210 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d0cc323515b340cbb1ea1249b47797e8: Sending DeleteTablet for 3 replicas of tablet e57ca99c02e041a1a96b5f6fb80e2b25
I20260924 00:56:03.007380 10628 tablet_service.cc:1514] Processing DeleteTablet for tablet e57ca99c02e041a1a96b5f6fb80e2b25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:56:03 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:60822
I20260924 00:56:03.007414 10349 tablet_service.cc:1514] Processing DeleteTablet for tablet e57ca99c02e041a1a96b5f6fb80e2b25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:56:03 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:43662
I20260924 00:56:03.008587 10489 tablet_service.cc:1514] Processing DeleteTablet for tablet e57ca99c02e041a1a96b5f6fb80e2b25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:56:03 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:34520
I20260924 00:56:03.013363 10210 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56768:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:56:03.014271 10210 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:56:03.022209 10488 tablet_service.cc:1467] Processing CreateTablet for tablet e83b657e4f884743b6effbf98abdf746 (DEFAULT_TABLE table=test-table [id=0763788558de4fa3a8aaf194f83917b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:56:03.022904 10488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e83b657e4f884743b6effbf98abdf746. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:03.027448 10627 tablet_service.cc:1467] Processing CreateTablet for tablet e83b657e4f884743b6effbf98abdf746 (DEFAULT_TABLE table=test-table [id=0763788558de4fa3a8aaf194f83917b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:56:03.027985 10627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e83b657e4f884743b6effbf98abdf746. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:03.028810 10348 tablet_service.cc:1467] Processing CreateTablet for tablet e83b657e4f884743b6effbf98abdf746 (DEFAULT_TABLE table=test-table [id=0763788558de4fa3a8aaf194f83917b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:56:03.029457 10348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e83b657e4f884743b6effbf98abdf746. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:03.032510 10767 tablet_replica.cc:330] T e57ca99c02e041a1a96b5f6fb80e2b25 P e51782766430416cba83a564995c41ba: stopping tablet replica
I20260924 00:56:03.033453 10720 tablet_bootstrap.cc:492] T e83b657e4f884743b6effbf98abdf746 P e51782766430416cba83a564995c41ba: Bootstrap starting.
I20260924 00:56:03.034015 10767 raft_consensus.cc:2229] T e57ca99c02e041a1a96b5f6fb80e2b25 P e51782766430416cba83a564995c41ba [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:56:03.034770 10767 raft_consensus.cc:2258] T e57ca99c02e041a1a96b5f6fb80e2b25 P e51782766430416cba83a564995c41ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:56:03.035463 10766 tablet_replica.cc:330] T e57ca99c02e041a1a96b5f6fb80e2b25 P 70748f17176b4448ad9cfbe5e1e7c33e: stopping tablet replica
I20260924 00:56:03.035566 10720 tablet_bootstrap.cc:654] T e83b657e4f884743b6effbf98abdf746 P e51782766430416cba83a564995c41ba: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:03.036306 10718 tablet_bootstrap.cc:492] T e83b657e4f884743b6effbf98abdf746 P 70748f17176b4448ad9cfbe5e1e7c33e: Bootstrap starting.
I20260924 00:56:03.036937 10766 raft_consensus.cc:2229] T e57ca99c02e041a1a96b5f6fb80e2b25 P 70748f17176b4448ad9cfbe5e1e7c33e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:56:03.037326 10766 raft_consensus.cc:2258] T e57ca99c02e041a1a96b5f6fb80e2b25 P 70748f17176b4448ad9cfbe5e1e7c33e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:56:03.038033 10718 tablet_bootstrap.cc:654] T e83b657e4f884743b6effbf98abdf746 P 70748f17176b4448ad9cfbe5e1e7c33e: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:03.038908 10767 ts_tablet_manager.cc:1905] T e57ca99c02e041a1a96b5f6fb80e2b25 P e51782766430416cba83a564995c41ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:56:03.039258 10717 tablet_bootstrap.cc:492] T e83b657e4f884743b6effbf98abdf746 P 08a0bd05c3b04ce594713e51cec5e82a: Bootstrap starting.
I20260924 00:56:03.041239 10717 tablet_bootstrap.cc:654] T e83b657e4f884743b6effbf98abdf746 P 08a0bd05c3b04ce594713e51cec5e82a: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:03.042744 10765 tablet_replica.cc:330] T e57ca99c02e041a1a96b5f6fb80e2b25 P 08a0bd05c3b04ce594713e51cec5e82a: stopping tablet replica
I20260924 00:56:03.044796 10766 ts_tablet_manager.cc:1905] T e57ca99c02e041a1a96b5f6fb80e2b25 P 70748f17176b4448ad9cfbe5e1e7c33e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:56:03.047851 10765 raft_consensus.cc:2229] T e57ca99c02e041a1a96b5f6fb80e2b25 P 08a0bd05c3b04ce594713e51cec5e82a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:56:03.048311 10765 raft_consensus.cc:2258] T e57ca99c02e041a1a96b5f6fb80e2b25 P 08a0bd05c3b04ce594713e51cec5e82a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:56:03.050480 10765 ts_tablet_manager.cc:1905] T e57ca99c02e041a1a96b5f6fb80e2b25 P 08a0bd05c3b04ce594713e51cec5e82a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:56:03.051208 10720 tablet_bootstrap.cc:492] T e83b657e4f884743b6effbf98abdf746 P e51782766430416cba83a564995c41ba: No bootstrap required, opened a new log
I20260924 00:56:03.051419 10720 ts_tablet_manager.cc:1397] T e83b657e4f884743b6effbf98abdf746 P e51782766430416cba83a564995c41ba: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.007s
I20260924 00:56:03.052302 10720 raft_consensus.cc:348] T e83b657e4f884743b6effbf98abdf746 P e51782766430416cba83a564995c41ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } }
I20260924 00:56:03.052561 10720 raft_consensus.cc:374] T e83b657e4f884743b6effbf98abdf746 P e51782766430416cba83a564995c41ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:03.052661 10720 raft_consensus.cc:729] T e83b657e4f884743b6effbf98abdf746 P e51782766430416cba83a564995c41ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51782766430416cba83a564995c41ba, State: Initialized, Role: FOLLOWER
I20260924 00:56:03.052994 10720 consensus_queue.cc:260] T e83b657e4f884743b6effbf98abdf746 P e51782766430416cba83a564995c41ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } }
I20260924 00:56:03.056360 10717 tablet_bootstrap.cc:492] T e83b657e4f884743b6effbf98abdf746 P 08a0bd05c3b04ce594713e51cec5e82a: No bootstrap required, opened a new log
I20260924 00:56:03.056573 10717 ts_tablet_manager.cc:1397] T e83b657e4f884743b6effbf98abdf746 P 08a0bd05c3b04ce594713e51cec5e82a: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.006s
I20260924 00:56:03.057374 10717 raft_consensus.cc:348] T e83b657e4f884743b6effbf98abdf746 P 08a0bd05c3b04ce594713e51cec5e82a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } }
I20260924 00:56:03.057626 10717 raft_consensus.cc:374] T e83b657e4f884743b6effbf98abdf746 P 08a0bd05c3b04ce594713e51cec5e82a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:03.057706 10717 raft_consensus.cc:729] T e83b657e4f884743b6effbf98abdf746 P 08a0bd05c3b04ce594713e51cec5e82a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08a0bd05c3b04ce594713e51cec5e82a, State: Initialized, Role: FOLLOWER
I20260924 00:56:03.057992 10717 consensus_queue.cc:260] T e83b657e4f884743b6effbf98abdf746 P 08a0bd05c3b04ce594713e51cec5e82a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } }
I20260924 00:56:03.059876 10720 ts_tablet_manager.cc:1428] T e83b657e4f884743b6effbf98abdf746 P e51782766430416cba83a564995c41ba: Time spent starting tablet: real 0.008s user 0.002s sys 0.001s
I20260924 00:56:03.060164 10718 tablet_bootstrap.cc:492] T e83b657e4f884743b6effbf98abdf746 P 70748f17176b4448ad9cfbe5e1e7c33e: No bootstrap required, opened a new log
I20260924 00:56:03.060351 10718 ts_tablet_manager.cc:1397] T e83b657e4f884743b6effbf98abdf746 P 70748f17176b4448ad9cfbe5e1e7c33e: Time spent bootstrapping tablet: real 0.024s user 0.006s sys 0.004s
I20260924 00:56:03.061002 10717 ts_tablet_manager.cc:1428] T e83b657e4f884743b6effbf98abdf746 P 08a0bd05c3b04ce594713e51cec5e82a: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260924 00:56:03.061141 10718 raft_consensus.cc:348] T e83b657e4f884743b6effbf98abdf746 P 70748f17176b4448ad9cfbe5e1e7c33e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } }
I20260924 00:56:03.061373 10718 raft_consensus.cc:374] T e83b657e4f884743b6effbf98abdf746 P 70748f17176b4448ad9cfbe5e1e7c33e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:03.061446 10718 raft_consensus.cc:729] T e83b657e4f884743b6effbf98abdf746 P 70748f17176b4448ad9cfbe5e1e7c33e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70748f17176b4448ad9cfbe5e1e7c33e, State: Initialized, Role: FOLLOWER
I20260924 00:56:03.061681 10718 consensus_queue.cc:260] T e83b657e4f884743b6effbf98abdf746 P 70748f17176b4448ad9cfbe5e1e7c33e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } }
I20260924 00:56:03.062273 10718 ts_tablet_manager.cc:1428] T e83b657e4f884743b6effbf98abdf746 P 70748f17176b4448ad9cfbe5e1e7c33e: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260924 00:56:03.064599 10765 ts_tablet_manager.cc:1918] T e57ca99c02e041a1a96b5f6fb80e2b25 P 08a0bd05c3b04ce594713e51cec5e82a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 00:56:03.064774 10765 log.cc:1196] T e57ca99c02e041a1a96b5f6fb80e2b25 P 08a0bd05c3b04ce594713e51cec5e82a: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-2/wal/wals/e57ca99c02e041a1a96b5f6fb80e2b25
I20260924 00:56:03.065303 10765 ts_tablet_manager.cc:1939] T e57ca99c02e041a1a96b5f6fb80e2b25 P 08a0bd05c3b04ce594713e51cec5e82a: Deleting consensus metadata
I20260924 00:56:03.067288 10767 ts_tablet_manager.cc:1918] T e57ca99c02e041a1a96b5f6fb80e2b25 P e51782766430416cba83a564995c41ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 00:56:03.067543 10767 log.cc:1196] T e57ca99c02e041a1a96b5f6fb80e2b25 P e51782766430416cba83a564995c41ba: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-1/wal/wals/e57ca99c02e041a1a96b5f6fb80e2b25
I20260924 00:56:03.067924 10195 catalog_manager.cc:4841] TS 08a0bd05c3b04ce594713e51cec5e82a (127.8.64.131:35983): tablet e57ca99c02e041a1a96b5f6fb80e2b25 (table test-table [id=64e8f0c20d0544b78b7f7b9cd06bcba2]) successfully deleted
I20260924 00:56:03.068257 10767 ts_tablet_manager.cc:1939] T e57ca99c02e041a1a96b5f6fb80e2b25 P e51782766430416cba83a564995c41ba: Deleting consensus metadata
I20260924 00:56:03.069921 10198 catalog_manager.cc:4841] TS e51782766430416cba83a564995c41ba (127.8.64.130:45425): tablet e57ca99c02e041a1a96b5f6fb80e2b25 (table test-table [id=64e8f0c20d0544b78b7f7b9cd06bcba2]) successfully deleted
I20260924 00:56:03.071435 10766 ts_tablet_manager.cc:1918] T e57ca99c02e041a1a96b5f6fb80e2b25 P 70748f17176b4448ad9cfbe5e1e7c33e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 00:56:03.071635 10766 log.cc:1196] T e57ca99c02e041a1a96b5f6fb80e2b25 P 70748f17176b4448ad9cfbe5e1e7c33e: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-0/wal/wals/e57ca99c02e041a1a96b5f6fb80e2b25
I20260924 00:56:03.072095 10766 ts_tablet_manager.cc:1939] T e57ca99c02e041a1a96b5f6fb80e2b25 P 70748f17176b4448ad9cfbe5e1e7c33e: Deleting consensus metadata
I20260924 00:56:03.073462 10197 catalog_manager.cc:4841] TS 70748f17176b4448ad9cfbe5e1e7c33e (127.8.64.129:46381): tablet e57ca99c02e041a1a96b5f6fb80e2b25 (table test-table [id=64e8f0c20d0544b78b7f7b9cd06bcba2]) successfully deleted
I20260924 00:56:03.336999 10723 raft_consensus.cc:482] T e83b657e4f884743b6effbf98abdf746 P 08a0bd05c3b04ce594713e51cec5e82a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:56:03.337255 10723 raft_consensus.cc:504] T e83b657e4f884743b6effbf98abdf746 P 08a0bd05c3b04ce594713e51cec5e82a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } }
I20260924 00:56:03.338881 10723 leader_election.cc:288] T e83b657e4f884743b6effbf98abdf746 P 08a0bd05c3b04ce594713e51cec5e82a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 70748f17176b4448ad9cfbe5e1e7c33e (127.8.64.129:46381), e51782766430416cba83a564995c41ba (127.8.64.130:45425)
I20260924 00:56:03.346985 10714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:03.339538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:50729 (local address 127.8.64.129:46381)
0924 00:56:03.339729 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:03.339735 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:03.340180 (+ 445us) server_negotiation.cc:407] Connection header received
0924 00:56:03.340516 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:03.340524 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:03.340599 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:03.340710 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:56:03.341428 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:03.342576 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:03.345212 (+ 2636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:03.345991 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:03.346206 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:03.346316 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:03.346585 (+ 269us) server_negotiation.cc:299] Negotiation successful
0924 00:56:03.346719 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260924 00:56:03.347071 10772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:03.339136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:46381 (local address 127.8.64.131:50729)
0924 00:56:03.340039 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:03.340070 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:03.340305 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:03.340728 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:03.340733 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:03.340766 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:56:03.341202 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:03.341217 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:03.342734 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:03.342739 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:03.344972 (+ 2233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:03.344984 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:03.346226 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:03.346232 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:03.346420 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:03.346447 (+ 27us) client_negotiation.cc:769] Sending connection context
0924 00:56:03.346558 (+ 111us) client_negotiation.cc:240] Negotiation successful
0924 00:56:03.346695 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":740,"mutex_wait_us":36,"thread_start_us":920,"threads_started":1}
I20260924 00:56:03.347890 10370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e83b657e4f884743b6effbf98abdf746" candidate_uuid: "08a0bd05c3b04ce594713e51cec5e82a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" is_pre_election: true
I20260924 00:56:03.348161 10370 raft_consensus.cc:2454] T e83b657e4f884743b6effbf98abdf746 P 70748f17176b4448ad9cfbe5e1e7c33e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08a0bd05c3b04ce594713e51cec5e82a in term 0.
I20260924 00:56:03.348326 10716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:03.339350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:58457 (local address 127.8.64.130:45425)
0924 00:56:03.339587 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:03.339595 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:03.340424 (+ 829us) server_negotiation.cc:407] Connection header received
0924 00:56:03.340557 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:03.340563 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:03.340643 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:03.340742 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:56:03.341464 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:03.342361 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:03.345169 (+ 2808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:03.346687 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:03.346901 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:03.346972 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:03.347891 (+ 919us) server_negotiation.cc:299] Negotiation successful
0924 00:56:03.348030 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260924 00:56:03.348357 10771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:03.339255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.130:45425 (local address 127.8.64.131:58457)
0924 00:56:03.340316 (+ 1061us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:03.340331 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:03.340436 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:03.340888 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:03.340892 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:03.340914 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:56:03.341332 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:03.341345 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:03.342543 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:03.342547 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:03.344968 (+ 2421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:03.344984 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:03.347471 (+ 2487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:03.347478 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:03.347685 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:03.347718 (+ 33us) client_negotiation.cc:769] Sending connection context
0924 00:56:03.347819 (+ 101us) client_negotiation.cc:240] Negotiation successful
0924 00:56:03.347969 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":954,"mutex_wait_us":46,"thread_start_us":482,"threads_started":1}
I20260924 00:56:03.348907 10581 leader_election.cc:302] T e83b657e4f884743b6effbf98abdf746 P 08a0bd05c3b04ce594713e51cec5e82a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08a0bd05c3b04ce594713e51cec5e82a, 70748f17176b4448ad9cfbe5e1e7c33e; no voters:
I20260924 00:56:03.349021 10514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e83b657e4f884743b6effbf98abdf746" candidate_uuid: "08a0bd05c3b04ce594713e51cec5e82a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51782766430416cba83a564995c41ba" is_pre_election: true
I20260924 00:56:03.349273 10514 raft_consensus.cc:2454] T e83b657e4f884743b6effbf98abdf746 P e51782766430416cba83a564995c41ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08a0bd05c3b04ce594713e51cec5e82a in term 0.
I20260924 00:56:03.349586 10723 raft_consensus.cc:2781] T e83b657e4f884743b6effbf98abdf746 P 08a0bd05c3b04ce594713e51cec5e82a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:56:03.349721 10723 raft_consensus.cc:482] T e83b657e4f884743b6effbf98abdf746 P 08a0bd05c3b04ce594713e51cec5e82a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:56:03.349834 10723 raft_consensus.cc:3037] T e83b657e4f884743b6effbf98abdf746 P 08a0bd05c3b04ce594713e51cec5e82a [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:03.352222 10723 raft_consensus.cc:504] T e83b657e4f884743b6effbf98abdf746 P 08a0bd05c3b04ce594713e51cec5e82a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } }
I20260924 00:56:03.352836 10723 leader_election.cc:288] T e83b657e4f884743b6effbf98abdf746 P 08a0bd05c3b04ce594713e51cec5e82a [CANDIDATE]: Term 1 election: Requested vote from peers 70748f17176b4448ad9cfbe5e1e7c33e (127.8.64.129:46381), e51782766430416cba83a564995c41ba (127.8.64.130:45425)
I20260924 00:56:03.353363 10370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e83b657e4f884743b6effbf98abdf746" candidate_uuid: "08a0bd05c3b04ce594713e51cec5e82a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70748f17176b4448ad9cfbe5e1e7c33e"
I20260924 00:56:03.353609 10370 raft_consensus.cc:3037] T e83b657e4f884743b6effbf98abdf746 P 70748f17176b4448ad9cfbe5e1e7c33e [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:03.353562 10514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e83b657e4f884743b6effbf98abdf746" candidate_uuid: "08a0bd05c3b04ce594713e51cec5e82a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51782766430416cba83a564995c41ba"
I20260924 00:56:03.353787 10514 raft_consensus.cc:3037] T e83b657e4f884743b6effbf98abdf746 P e51782766430416cba83a564995c41ba [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:03.355698 10370 raft_consensus.cc:2454] T e83b657e4f884743b6effbf98abdf746 P 70748f17176b4448ad9cfbe5e1e7c33e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08a0bd05c3b04ce594713e51cec5e82a in term 1.
I20260924 00:56:03.356115 10581 leader_election.cc:302] T e83b657e4f884743b6effbf98abdf746 P 08a0bd05c3b04ce594713e51cec5e82a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08a0bd05c3b04ce594713e51cec5e82a, 70748f17176b4448ad9cfbe5e1e7c33e; no voters:
I20260924 00:56:03.356384 10723 raft_consensus.cc:2781] T e83b657e4f884743b6effbf98abdf746 P 08a0bd05c3b04ce594713e51cec5e82a [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:56:03.356890 10723 raft_consensus.cc:686] T e83b657e4f884743b6effbf98abdf746 P 08a0bd05c3b04ce594713e51cec5e82a [term 1 LEADER]: Becoming Leader. State: Replica: 08a0bd05c3b04ce594713e51cec5e82a, State: Running, Role: LEADER
I20260924 00:56:03.357084 10514 raft_consensus.cc:2454] T e83b657e4f884743b6effbf98abdf746 P e51782766430416cba83a564995c41ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08a0bd05c3b04ce594713e51cec5e82a in term 1.
I20260924 00:56:03.357518 10723 consensus_queue.cc:237] T e83b657e4f884743b6effbf98abdf746 P 08a0bd05c3b04ce594713e51cec5e82a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } }
I20260924 00:56:03.361764 10210 catalog_manager.cc:5495] T e83b657e4f884743b6effbf98abdf746 P 08a0bd05c3b04ce594713e51cec5e82a reported cstate change: term changed from 0 to 1, leader changed from <none> to 08a0bd05c3b04ce594713e51cec5e82a (127.8.64.131). New cstate: current_term: 1 leader_uuid: "08a0bd05c3b04ce594713e51cec5e82a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } health_report { overall_health: HEALTHY } } }
I20260924 00:56:03.422879 10777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:03.414902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60836 (local address 127.8.64.131:35983)
0924 00:56:03.416259 (+ 1357us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:03.416264 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:03.416284 (+ 20us) server_negotiation.cc:407] Connection header received
0924 00:56:03.416367 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:03.416371 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:03.416473 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:03.416621 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:56:03.417234 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:03.418174 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:03.420269 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:03.420745 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:03.420849 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:03.421385 (+ 536us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:56:03.422079 (+ 694us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:56:03.422191 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:03.422429 (+ 238us) server_negotiation.cc:299] Negotiation successful
0924 00:56:03.422495 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1254,"thread_start_us":1122,"threads_started":1}
I20260924 00:56:03.439301 10714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:03.433827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:50619 (local address 127.8.64.129:46381)
0924 00:56:03.434038 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:03.434045 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:03.434538 (+ 493us) server_negotiation.cc:407] Connection header received
0924 00:56:03.434741 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:03.434746 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:03.434855 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:03.434977 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:56:03.435558 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:03.436314 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:03.437622 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:03.438012 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:03.438150 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:03.438631 (+ 481us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:56:03.438732 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:56:03.438830 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:03.438936 (+ 106us) server_negotiation.cc:299] Negotiation successful
0924 00:56:03.439009 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260924 00:56:03.439410 10780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:03.433697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:46381 (local address 127.8.64.131:50619)
0924 00:56:03.434430 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:03.434447 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:03.434608 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:03.434973 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:03.434978 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:03.434990 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 00:56:03.435397 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:03.435406 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:03.436441 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:03.436449 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:03.437482 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:03.437492 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:03.438182 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:03.438190 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:03.438391 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:03.438440 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 00:56:03.438830 (+ 390us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 00:56:03.438843 (+ 13us) client_negotiation.cc:769] Sending connection context
0924 00:56:03.438919 (+ 76us) client_negotiation.cc:240] Negotiation successful
0924 00:56:03.439052 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":262,"threads_started":1}
I20260924 00:56:03.448114 10514 raft_consensus.cc:1261] T e83b657e4f884743b6effbf98abdf746 P e51782766430416cba83a564995c41ba [term 1 FOLLOWER]: Refusing update from remote peer 08a0bd05c3b04ce594713e51cec5e82a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:56:03.448105 10370 raft_consensus.cc:1261] T e83b657e4f884743b6effbf98abdf746 P 70748f17176b4448ad9cfbe5e1e7c33e [term 1 FOLLOWER]: Refusing update from remote peer 08a0bd05c3b04ce594713e51cec5e82a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:56:03.448822 10723 consensus_queue.cc:1035] T e83b657e4f884743b6effbf98abdf746 P 08a0bd05c3b04ce594713e51cec5e82a [LEADER]: Connected to new peer: Peer: permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:56:03.449111 10773 consensus_queue.cc:1035] T e83b657e4f884743b6effbf98abdf746 P 08a0bd05c3b04ce594713e51cec5e82a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:56:03.480645 10777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:03.474524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:58463 (local address 127.8.64.131:35983)
0924 00:56:03.474874 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:03.474880 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:03.475066 (+ 186us) server_negotiation.cc:407] Connection header received
0924 00:56:03.475263 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:03.475268 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:03.475343 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:03.475436 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:56:03.475809 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:03.476763 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:03.478757 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:03.479330 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:03.479411 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:03.479877 (+ 466us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:56:03.479986 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:56:03.480086 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:03.480239 (+ 153us) server_negotiation.cc:299] Negotiation successful
0924 00:56:03.480315 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219}
I20260924 00:56:03.480675 10784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:03.474452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.131:35983 (local address 127.8.64.129:58463)
0924 00:56:03.474955 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:03.474982 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:03.475132 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:03.475450 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:03.475454 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:03.475465 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 00:56:03.475665 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:03.475674 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:03.476882 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:03.476889 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:03.478574 (+ 1685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:03.478592 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:03.479540 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:03.479545 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:03.479664 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:03.479697 (+ 33us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 00:56:03.480088 (+ 391us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 00:56:03.480102 (+ 14us) client_negotiation.cc:769] Sending connection context
0924 00:56:03.480219 (+ 117us) client_negotiation.cc:240] Negotiation successful
0924 00:56:03.480356 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":256,"threads_started":1}
W20260924 00:56:03.481773 10748 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260924 00:56:05.080423 10211 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:56:05.080641 10211 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:56:05.084438 10211 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d0cc323515b340cbb1ea1249b47797e8: Sending DeleteTablet for 3 replicas of tablet e83b657e4f884743b6effbf98abdf746
I20260924 00:56:05.085193 10349 tablet_service.cc:1514] Processing DeleteTablet for tablet e83b657e4f884743b6effbf98abdf746 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:56:05 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:43662
I20260924 00:56:05.085263 10488 tablet_service.cc:1514] Processing DeleteTablet for tablet e83b657e4f884743b6effbf98abdf746 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:56:05 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:34520
I20260924 00:56:05.085453 10627 tablet_service.cc:1514] Processing DeleteTablet for tablet e83b657e4f884743b6effbf98abdf746 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:56:05 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:60822
I20260924 00:56:05.100576 10811 tablet_replica.cc:330] T e83b657e4f884743b6effbf98abdf746 P 70748f17176b4448ad9cfbe5e1e7c33e: stopping tablet replica
I20260924 00:56:05.100858 10811 raft_consensus.cc:2229] T e83b657e4f884743b6effbf98abdf746 P 70748f17176b4448ad9cfbe5e1e7c33e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:56:05.101130 10811 raft_consensus.cc:2258] T e83b657e4f884743b6effbf98abdf746 P 70748f17176b4448ad9cfbe5e1e7c33e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:56:05.101651 10813 tablet_replica.cc:330] T e83b657e4f884743b6effbf98abdf746 P 08a0bd05c3b04ce594713e51cec5e82a: stopping tablet replica
I20260924 00:56:05.101794 10812 tablet_replica.cc:330] T e83b657e4f884743b6effbf98abdf746 P e51782766430416cba83a564995c41ba: stopping tablet replica
I20260924 00:56:05.102021 10813 raft_consensus.cc:2229] T e83b657e4f884743b6effbf98abdf746 P 08a0bd05c3b04ce594713e51cec5e82a [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:56:05.102056 10812 raft_consensus.cc:2229] T e83b657e4f884743b6effbf98abdf746 P e51782766430416cba83a564995c41ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:56:05.102368 10812 raft_consensus.cc:2258] T e83b657e4f884743b6effbf98abdf746 P e51782766430416cba83a564995c41ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:56:05.102679 10811 ts_tablet_manager.cc:1905] T e83b657e4f884743b6effbf98abdf746 P 70748f17176b4448ad9cfbe5e1e7c33e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:56:05.103245 10813 raft_consensus.cc:2258] T e83b657e4f884743b6effbf98abdf746 P 08a0bd05c3b04ce594713e51cec5e82a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:56:05.103453 10812 ts_tablet_manager.cc:1905] T e83b657e4f884743b6effbf98abdf746 P e51782766430416cba83a564995c41ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:56:05.104264 10813 ts_tablet_manager.cc:1905] T e83b657e4f884743b6effbf98abdf746 P 08a0bd05c3b04ce594713e51cec5e82a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:56:05.109332 10811 ts_tablet_manager.cc:1918] T e83b657e4f884743b6effbf98abdf746 P 70748f17176b4448ad9cfbe5e1e7c33e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260924 00:56:05.109592 10811 log.cc:1196] T e83b657e4f884743b6effbf98abdf746 P 70748f17176b4448ad9cfbe5e1e7c33e: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-0/wal/wals/e83b657e4f884743b6effbf98abdf746
I20260924 00:56:05.110131 10811 ts_tablet_manager.cc:1939] T e83b657e4f884743b6effbf98abdf746 P 70748f17176b4448ad9cfbe5e1e7c33e: Deleting consensus metadata
I20260924 00:56:05.111050 10813 ts_tablet_manager.cc:1918] T e83b657e4f884743b6effbf98abdf746 P 08a0bd05c3b04ce594713e51cec5e82a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260924 00:56:05.111081 10812 ts_tablet_manager.cc:1918] T e83b657e4f884743b6effbf98abdf746 P e51782766430416cba83a564995c41ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260924 00:56:05.111258 10812 log.cc:1196] T e83b657e4f884743b6effbf98abdf746 P e51782766430416cba83a564995c41ba: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-1/wal/wals/e83b657e4f884743b6effbf98abdf746
I20260924 00:56:05.111287 10813 log.cc:1196] T e83b657e4f884743b6effbf98abdf746 P 08a0bd05c3b04ce594713e51cec5e82a: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-2/wal/wals/e83b657e4f884743b6effbf98abdf746
I20260924 00:56:05.111536 10197 catalog_manager.cc:4841] TS 70748f17176b4448ad9cfbe5e1e7c33e (127.8.64.129:46381): tablet e83b657e4f884743b6effbf98abdf746 (table test-table [id=0763788558de4fa3a8aaf194f83917b1]) successfully deleted
I20260924 00:56:05.111765 10812 ts_tablet_manager.cc:1939] T e83b657e4f884743b6effbf98abdf746 P e51782766430416cba83a564995c41ba: Deleting consensus metadata
I20260924 00:56:05.111917 10813 ts_tablet_manager.cc:1939] T e83b657e4f884743b6effbf98abdf746 P 08a0bd05c3b04ce594713e51cec5e82a: Deleting consensus metadata
I20260924 00:56:05.112907 10198 catalog_manager.cc:4841] TS e51782766430416cba83a564995c41ba (127.8.64.130:45425): tablet e83b657e4f884743b6effbf98abdf746 (table test-table [id=0763788558de4fa3a8aaf194f83917b1]) successfully deleted
I20260924 00:56:05.113066 10195 catalog_manager.cc:4841] TS 08a0bd05c3b04ce594713e51cec5e82a (127.8.64.131:35983): tablet e83b657e4f884743b6effbf98abdf746 (table test-table [id=0763788558de4fa3a8aaf194f83917b1]) successfully deleted
Sep 24 00:56:05 dist-test-slave-mmhs krb5kdc[10166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211365, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 24 00:56:05 dist-test-slave-mmhs krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211365, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:56:05.147156 10821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:05.132978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51502 (local address 127.8.64.190:44123)
0924 00:56:05.133640 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:05.133644 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:05.133667 (+ 23us) server_negotiation.cc:407] Connection header received
0924 00:56:05.133741 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:05.133745 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:05.133805 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:05.133892 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:05.135281 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:05.136090 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:05.137327 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:05.137727 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:05.137841 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:05.141948 (+ 4107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:05.141953 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:05.141956 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:05.141987 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:05.144797 (+ 2810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:05.145410 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:05.145414 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:05.145416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:05.145535 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:05.145851 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:05.145854 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:05.145856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:05.146094 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:05.146285 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:05.146593 (+ 308us) server_negotiation.cc:299] Negotiation successful
0924 00:56:05.146758 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514,"thread_start_us":336,"threads_started":1}
I20260924 00:56:05.150369 10211 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51502:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:56:05.151362 10211 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:56:05.157634 10488 tablet_service.cc:1467] Processing CreateTablet for tablet 1865004f84854975a4284f520c892b4e (DEFAULT_TABLE table=test-table [id=7df2acd7d29f4653be67c46ed121fce8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:56:05.157740 10349 tablet_service.cc:1467] Processing CreateTablet for tablet 1865004f84854975a4284f520c892b4e (DEFAULT_TABLE table=test-table [id=7df2acd7d29f4653be67c46ed121fce8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:56:05.157876 10627 tablet_service.cc:1467] Processing CreateTablet for tablet 1865004f84854975a4284f520c892b4e (DEFAULT_TABLE table=test-table [id=7df2acd7d29f4653be67c46ed121fce8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:56:05.158094 10488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1865004f84854975a4284f520c892b4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:05.158209 10349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1865004f84854975a4284f520c892b4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:05.158319 10627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1865004f84854975a4284f520c892b4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:05.164389 10822 tablet_bootstrap.cc:492] T 1865004f84854975a4284f520c892b4e P 70748f17176b4448ad9cfbe5e1e7c33e: Bootstrap starting.
I20260924 00:56:05.165030 10823 tablet_bootstrap.cc:492] T 1865004f84854975a4284f520c892b4e P e51782766430416cba83a564995c41ba: Bootstrap starting.
I20260924 00:56:05.165853 10824 tablet_bootstrap.cc:492] T 1865004f84854975a4284f520c892b4e P 08a0bd05c3b04ce594713e51cec5e82a: Bootstrap starting.
I20260924 00:56:05.166762 10822 tablet_bootstrap.cc:654] T 1865004f84854975a4284f520c892b4e P 70748f17176b4448ad9cfbe5e1e7c33e: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:05.167222 10823 tablet_bootstrap.cc:654] T 1865004f84854975a4284f520c892b4e P e51782766430416cba83a564995c41ba: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:05.167740 10824 tablet_bootstrap.cc:654] T 1865004f84854975a4284f520c892b4e P 08a0bd05c3b04ce594713e51cec5e82a: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:05.168964 10822 tablet_bootstrap.cc:492] T 1865004f84854975a4284f520c892b4e P 70748f17176b4448ad9cfbe5e1e7c33e: No bootstrap required, opened a new log
I20260924 00:56:05.169113 10822 ts_tablet_manager.cc:1397] T 1865004f84854975a4284f520c892b4e P 70748f17176b4448ad9cfbe5e1e7c33e: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260924 00:56:05.169767 10824 tablet_bootstrap.cc:492] T 1865004f84854975a4284f520c892b4e P 08a0bd05c3b04ce594713e51cec5e82a: No bootstrap required, opened a new log
I20260924 00:56:05.169740 10822 raft_consensus.cc:348] T 1865004f84854975a4284f520c892b4e P 70748f17176b4448ad9cfbe5e1e7c33e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } }
I20260924 00:56:05.169994 10824 ts_tablet_manager.cc:1397] T 1865004f84854975a4284f520c892b4e P 08a0bd05c3b04ce594713e51cec5e82a: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260924 00:56:05.170017 10822 raft_consensus.cc:374] T 1865004f84854975a4284f520c892b4e P 70748f17176b4448ad9cfbe5e1e7c33e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:05.170135 10822 raft_consensus.cc:729] T 1865004f84854975a4284f520c892b4e P 70748f17176b4448ad9cfbe5e1e7c33e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70748f17176b4448ad9cfbe5e1e7c33e, State: Initialized, Role: FOLLOWER
I20260924 00:56:05.170408 10822 consensus_queue.cc:260] T 1865004f84854975a4284f520c892b4e P 70748f17176b4448ad9cfbe5e1e7c33e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } }
I20260924 00:56:05.170895 10822 ts_tablet_manager.cc:1428] T 1865004f84854975a4284f520c892b4e P 70748f17176b4448ad9cfbe5e1e7c33e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 00:56:05.170816 10824 raft_consensus.cc:348] T 1865004f84854975a4284f520c892b4e P 08a0bd05c3b04ce594713e51cec5e82a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } }
I20260924 00:56:05.171043 10824 raft_consensus.cc:374] T 1865004f84854975a4284f520c892b4e P 08a0bd05c3b04ce594713e51cec5e82a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:05.171147 10824 raft_consensus.cc:729] T 1865004f84854975a4284f520c892b4e P 08a0bd05c3b04ce594713e51cec5e82a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08a0bd05c3b04ce594713e51cec5e82a, State: Initialized, Role: FOLLOWER
I20260924 00:56:05.171381 10824 consensus_queue.cc:260] T 1865004f84854975a4284f520c892b4e P 08a0bd05c3b04ce594713e51cec5e82a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } }
I20260924 00:56:05.171725 10823 tablet_bootstrap.cc:492] T 1865004f84854975a4284f520c892b4e P e51782766430416cba83a564995c41ba: No bootstrap required, opened a new log
I20260924 00:56:05.171857 10823 ts_tablet_manager.cc:1397] T 1865004f84854975a4284f520c892b4e P e51782766430416cba83a564995c41ba: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260924 00:56:05.172039 10824 ts_tablet_manager.cc:1428] T 1865004f84854975a4284f520c892b4e P 08a0bd05c3b04ce594713e51cec5e82a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 00:56:05.172484 10823 raft_consensus.cc:348] T 1865004f84854975a4284f520c892b4e P e51782766430416cba83a564995c41ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } }
I20260924 00:56:05.172742 10823 raft_consensus.cc:374] T 1865004f84854975a4284f520c892b4e P e51782766430416cba83a564995c41ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:05.173015 10823 raft_consensus.cc:729] T 1865004f84854975a4284f520c892b4e P e51782766430416cba83a564995c41ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51782766430416cba83a564995c41ba, State: Initialized, Role: FOLLOWER
I20260924 00:56:05.173269 10823 consensus_queue.cc:260] T 1865004f84854975a4284f520c892b4e P e51782766430416cba83a564995c41ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } }
I20260924 00:56:05.174086 10823 ts_tablet_manager.cc:1428] T 1865004f84854975a4284f520c892b4e P e51782766430416cba83a564995c41ba: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260924 00:56:05.340107 10828 raft_consensus.cc:482] T 1865004f84854975a4284f520c892b4e P e51782766430416cba83a564995c41ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:56:05.340329 10828 raft_consensus.cc:504] T 1865004f84854975a4284f520c892b4e P e51782766430416cba83a564995c41ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } }
I20260924 00:56:05.340988 10828 leader_election.cc:288] T 1865004f84854975a4284f520c892b4e P e51782766430416cba83a564995c41ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 70748f17176b4448ad9cfbe5e1e7c33e (127.8.64.129:46381), 08a0bd05c3b04ce594713e51cec5e82a (127.8.64.131:35983)
I20260924 00:56:05.341542 10370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1865004f84854975a4284f520c892b4e" candidate_uuid: "e51782766430416cba83a564995c41ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" is_pre_election: true
I20260924 00:56:05.341569 10647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1865004f84854975a4284f520c892b4e" candidate_uuid: "e51782766430416cba83a564995c41ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08a0bd05c3b04ce594713e51cec5e82a" is_pre_election: true
I20260924 00:56:05.341806 10370 raft_consensus.cc:2454] T 1865004f84854975a4284f520c892b4e P 70748f17176b4448ad9cfbe5e1e7c33e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51782766430416cba83a564995c41ba in term 0.
I20260924 00:56:05.341826 10647 raft_consensus.cc:2454] T 1865004f84854975a4284f520c892b4e P 08a0bd05c3b04ce594713e51cec5e82a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51782766430416cba83a564995c41ba in term 0.
I20260924 00:56:05.342283 10442 leader_election.cc:302] T 1865004f84854975a4284f520c892b4e P e51782766430416cba83a564995c41ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70748f17176b4448ad9cfbe5e1e7c33e, e51782766430416cba83a564995c41ba; no voters:
I20260924 00:56:05.342573 10828 raft_consensus.cc:2781] T 1865004f84854975a4284f520c892b4e P e51782766430416cba83a564995c41ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:56:05.342671 10828 raft_consensus.cc:482] T 1865004f84854975a4284f520c892b4e P e51782766430416cba83a564995c41ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:56:05.342782 10828 raft_consensus.cc:3037] T 1865004f84854975a4284f520c892b4e P e51782766430416cba83a564995c41ba [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:05.345188 10828 raft_consensus.cc:504] T 1865004f84854975a4284f520c892b4e P e51782766430416cba83a564995c41ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } }
I20260924 00:56:05.345719 10828 leader_election.cc:288] T 1865004f84854975a4284f520c892b4e P e51782766430416cba83a564995c41ba [CANDIDATE]: Term 1 election: Requested vote from peers 70748f17176b4448ad9cfbe5e1e7c33e (127.8.64.129:46381), 08a0bd05c3b04ce594713e51cec5e82a (127.8.64.131:35983)
I20260924 00:56:05.346153 10370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1865004f84854975a4284f520c892b4e" candidate_uuid: "e51782766430416cba83a564995c41ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70748f17176b4448ad9cfbe5e1e7c33e"
I20260924 00:56:05.346218 10647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1865004f84854975a4284f520c892b4e" candidate_uuid: "e51782766430416cba83a564995c41ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08a0bd05c3b04ce594713e51cec5e82a"
I20260924 00:56:05.346377 10370 raft_consensus.cc:3037] T 1865004f84854975a4284f520c892b4e P 70748f17176b4448ad9cfbe5e1e7c33e [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:05.346499 10647 raft_consensus.cc:3037] T 1865004f84854975a4284f520c892b4e P 08a0bd05c3b04ce594713e51cec5e82a [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:05.348333 10370 raft_consensus.cc:2454] T 1865004f84854975a4284f520c892b4e P 70748f17176b4448ad9cfbe5e1e7c33e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51782766430416cba83a564995c41ba in term 1.
I20260924 00:56:05.348531 10647 raft_consensus.cc:2454] T 1865004f84854975a4284f520c892b4e P 08a0bd05c3b04ce594713e51cec5e82a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51782766430416cba83a564995c41ba in term 1.
I20260924 00:56:05.348739 10442 leader_election.cc:302] T 1865004f84854975a4284f520c892b4e P e51782766430416cba83a564995c41ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70748f17176b4448ad9cfbe5e1e7c33e, e51782766430416cba83a564995c41ba; no voters:
I20260924 00:56:05.349040 10828 raft_consensus.cc:2781] T 1865004f84854975a4284f520c892b4e P e51782766430416cba83a564995c41ba [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:56:05.349515 10828 raft_consensus.cc:686] T 1865004f84854975a4284f520c892b4e P e51782766430416cba83a564995c41ba [term 1 LEADER]: Becoming Leader. State: Replica: e51782766430416cba83a564995c41ba, State: Running, Role: LEADER
I20260924 00:56:05.349807 10828 consensus_queue.cc:237] T 1865004f84854975a4284f520c892b4e P e51782766430416cba83a564995c41ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } }
I20260924 00:56:05.352087 10210 catalog_manager.cc:5495] T 1865004f84854975a4284f520c892b4e P e51782766430416cba83a564995c41ba reported cstate change: term changed from 0 to 1, leader changed from <none> to e51782766430416cba83a564995c41ba (127.8.64.130). New cstate: current_term: 1 leader_uuid: "e51782766430416cba83a564995c41ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } health_report { overall_health: UNKNOWN } } }
I20260924 00:56:05.412936 10832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:05.407818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48122 (local address 127.8.64.130:45425)
0924 00:56:05.408362 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:05.408366 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:05.408383 (+ 17us) server_negotiation.cc:407] Connection header received
0924 00:56:05.408429 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:05.408432 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:05.408486 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:05.408601 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:56:05.409004 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:05.409762 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:05.411217 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:05.411660 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:05.411795 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:05.412209 (+ 414us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:56:05.412314 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:56:05.412412 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:05.412573 (+ 161us) server_negotiation.cc:299] Negotiation successful
0924 00:56:05.412645 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":281,"threads_started":1}
I20260924 00:56:05.420805 10647 raft_consensus.cc:1261] T 1865004f84854975a4284f520c892b4e P 08a0bd05c3b04ce594713e51cec5e82a [term 1 FOLLOWER]: Refusing update from remote peer e51782766430416cba83a564995c41ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:56:05.421046 10370 raft_consensus.cc:1261] T 1865004f84854975a4284f520c892b4e P 70748f17176b4448ad9cfbe5e1e7c33e [term 1 FOLLOWER]: Refusing update from remote peer e51782766430416cba83a564995c41ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:56:05.421310 10828 consensus_queue.cc:1035] T 1865004f84854975a4284f520c892b4e P e51782766430416cba83a564995c41ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:56:05.421610 10829 consensus_queue.cc:1035] T 1865004f84854975a4284f520c892b4e P e51782766430416cba83a564995c41ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:56:05.461755 10211 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51502:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:56:05.461999 10211 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51502:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:56:05.464692 10211 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d0cc323515b340cbb1ea1249b47797e8: Sending DeleteTablet for 3 replicas of tablet 1865004f84854975a4284f520c892b4e
I20260924 00:56:05.465408 10349 tablet_service.cc:1514] Processing DeleteTablet for tablet 1865004f84854975a4284f520c892b4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:56:05 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:43662
I20260924 00:56:05.465492 10488 tablet_service.cc:1514] Processing DeleteTablet for tablet 1865004f84854975a4284f520c892b4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:56:05 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:34520
I20260924 00:56:05.465544 10627 tablet_service.cc:1514] Processing DeleteTablet for tablet 1865004f84854975a4284f520c892b4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:56:05 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:60822
I20260924 00:56:05.465777 10811 tablet_replica.cc:330] T 1865004f84854975a4284f520c892b4e P 70748f17176b4448ad9cfbe5e1e7c33e: stopping tablet replica
I20260924 00:56:05.465777 10812 tablet_replica.cc:330] T 1865004f84854975a4284f520c892b4e P e51782766430416cba83a564995c41ba: stopping tablet replica
I20260924 00:56:05.465787 10813 tablet_replica.cc:330] T 1865004f84854975a4284f520c892b4e P 08a0bd05c3b04ce594713e51cec5e82a: stopping tablet replica
I20260924 00:56:05.465981 10811 raft_consensus.cc:2229] T 1865004f84854975a4284f520c892b4e P 70748f17176b4448ad9cfbe5e1e7c33e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:56:05.465988 10812 raft_consensus.cc:2229] T 1865004f84854975a4284f520c892b4e P e51782766430416cba83a564995c41ba [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:56:05.466048 10813 raft_consensus.cc:2229] T 1865004f84854975a4284f520c892b4e P 08a0bd05c3b04ce594713e51cec5e82a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:56:05.466159 10811 raft_consensus.cc:2258] T 1865004f84854975a4284f520c892b4e P 70748f17176b4448ad9cfbe5e1e7c33e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:56:05.466267 10813 raft_consensus.cc:2258] T 1865004f84854975a4284f520c892b4e P 08a0bd05c3b04ce594713e51cec5e82a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:56:05.466279 10812 raft_consensus.cc:2258] T 1865004f84854975a4284f520c892b4e P e51782766430416cba83a564995c41ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:56:05.467363 10812 ts_tablet_manager.cc:1905] T 1865004f84854975a4284f520c892b4e P e51782766430416cba83a564995c41ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:56:05.467507 10813 ts_tablet_manager.cc:1905] T 1865004f84854975a4284f520c892b4e P 08a0bd05c3b04ce594713e51cec5e82a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:56:05.467471 10211 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51502:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:56:05.467638 10811 ts_tablet_manager.cc:1905] T 1865004f84854975a4284f520c892b4e P 70748f17176b4448ad9cfbe5e1e7c33e: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260924 00:56:05.468039 10211 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:56:05.473412 10812 ts_tablet_manager.cc:1918] T 1865004f84854975a4284f520c892b4e P e51782766430416cba83a564995c41ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 00:56:05.473567 10812 log.cc:1196] T 1865004f84854975a4284f520c892b4e P e51782766430416cba83a564995c41ba: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-1/wal/wals/1865004f84854975a4284f520c892b4e
I20260924 00:56:05.473839 10813 ts_tablet_manager.cc:1918] T 1865004f84854975a4284f520c892b4e P 08a0bd05c3b04ce594713e51cec5e82a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 00:56:05.473975 10813 log.cc:1196] T 1865004f84854975a4284f520c892b4e P 08a0bd05c3b04ce594713e51cec5e82a: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-2/wal/wals/1865004f84854975a4284f520c892b4e
I20260924 00:56:05.474056 10812 ts_tablet_manager.cc:1939] T 1865004f84854975a4284f520c892b4e P e51782766430416cba83a564995c41ba: Deleting consensus metadata
I20260924 00:56:05.474472 10813 ts_tablet_manager.cc:1939] T 1865004f84854975a4284f520c892b4e P 08a0bd05c3b04ce594713e51cec5e82a: Deleting consensus metadata
I20260924 00:56:05.475243 10198 catalog_manager.cc:4841] TS e51782766430416cba83a564995c41ba (127.8.64.130:45425): tablet 1865004f84854975a4284f520c892b4e (table test-table [id=7df2acd7d29f4653be67c46ed121fce8]) successfully deleted
I20260924 00:56:05.475353 10488 tablet_service.cc:1467] Processing CreateTablet for tablet 53cfcbe0257d44a283fcfc7b3fa42d88 (DEFAULT_TABLE table=test-table [id=a7e1137a0e204f5ea4580995e32f467f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:56:05.475677 10627 tablet_service.cc:1467] Processing CreateTablet for tablet 53cfcbe0257d44a283fcfc7b3fa42d88 (DEFAULT_TABLE table=test-table [id=a7e1137a0e204f5ea4580995e32f467f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:56:05.475744 10488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53cfcbe0257d44a283fcfc7b3fa42d88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:05.475829 10349 tablet_service.cc:1467] Processing CreateTablet for tablet 53cfcbe0257d44a283fcfc7b3fa42d88 (DEFAULT_TABLE table=test-table [id=a7e1137a0e204f5ea4580995e32f467f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:56:05.476047 10627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53cfcbe0257d44a283fcfc7b3fa42d88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:05.476284 10349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53cfcbe0257d44a283fcfc7b3fa42d88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:05.480863 10824 tablet_bootstrap.cc:492] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 08a0bd05c3b04ce594713e51cec5e82a: Bootstrap starting.
I20260924 00:56:05.481032 10195 catalog_manager.cc:4841] TS 08a0bd05c3b04ce594713e51cec5e82a (127.8.64.131:35983): tablet 1865004f84854975a4284f520c892b4e (table test-table [id=7df2acd7d29f4653be67c46ed121fce8]) successfully deleted
I20260924 00:56:05.482254 10823 tablet_bootstrap.cc:492] T 53cfcbe0257d44a283fcfc7b3fa42d88 P e51782766430416cba83a564995c41ba: Bootstrap starting.
I20260924 00:56:05.482545 10824 tablet_bootstrap.cc:654] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 08a0bd05c3b04ce594713e51cec5e82a: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:05.482918 10822 tablet_bootstrap.cc:492] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 70748f17176b4448ad9cfbe5e1e7c33e: Bootstrap starting.
I20260924 00:56:05.483887 10823 tablet_bootstrap.cc:654] T 53cfcbe0257d44a283fcfc7b3fa42d88 P e51782766430416cba83a564995c41ba: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:05.484659 10822 tablet_bootstrap.cc:654] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 70748f17176b4448ad9cfbe5e1e7c33e: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:05.486045 10811 ts_tablet_manager.cc:1918] T 1865004f84854975a4284f520c892b4e P 70748f17176b4448ad9cfbe5e1e7c33e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 00:56:05.486276 10811 log.cc:1196] T 1865004f84854975a4284f520c892b4e P 70748f17176b4448ad9cfbe5e1e7c33e: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-0/wal/wals/1865004f84854975a4284f520c892b4e
I20260924 00:56:05.488243 10811 ts_tablet_manager.cc:1939] T 1865004f84854975a4284f520c892b4e P 70748f17176b4448ad9cfbe5e1e7c33e: Deleting consensus metadata
I20260924 00:56:05.489801 10197 catalog_manager.cc:4841] TS 70748f17176b4448ad9cfbe5e1e7c33e (127.8.64.129:46381): tablet 1865004f84854975a4284f520c892b4e (table test-table [id=7df2acd7d29f4653be67c46ed121fce8]) successfully deleted
I20260924 00:56:05.494439 10824 tablet_bootstrap.cc:492] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 08a0bd05c3b04ce594713e51cec5e82a: No bootstrap required, opened a new log
I20260924 00:56:05.494633 10824 ts_tablet_manager.cc:1397] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 08a0bd05c3b04ce594713e51cec5e82a: Time spent bootstrapping tablet: real 0.014s user 0.013s sys 0.000s
I20260924 00:56:05.495174 10822 tablet_bootstrap.cc:492] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 70748f17176b4448ad9cfbe5e1e7c33e: No bootstrap required, opened a new log
I20260924 00:56:05.495208 10823 tablet_bootstrap.cc:492] T 53cfcbe0257d44a283fcfc7b3fa42d88 P e51782766430416cba83a564995c41ba: No bootstrap required, opened a new log
I20260924 00:56:05.495339 10823 ts_tablet_manager.cc:1397] T 53cfcbe0257d44a283fcfc7b3fa42d88 P e51782766430416cba83a564995c41ba: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.007s
I20260924 00:56:05.495339 10822 ts_tablet_manager.cc:1397] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 70748f17176b4448ad9cfbe5e1e7c33e: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.008s
I20260924 00:56:05.495426 10824 raft_consensus.cc:348] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 08a0bd05c3b04ce594713e51cec5e82a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } }
I20260924 00:56:05.495693 10824 raft_consensus.cc:374] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 08a0bd05c3b04ce594713e51cec5e82a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:05.495772 10824 raft_consensus.cc:729] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 08a0bd05c3b04ce594713e51cec5e82a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08a0bd05c3b04ce594713e51cec5e82a, State: Initialized, Role: FOLLOWER
I20260924 00:56:05.496007 10824 consensus_queue.cc:260] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 08a0bd05c3b04ce594713e51cec5e82a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } }
I20260924 00:56:05.496065 10822 raft_consensus.cc:348] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 70748f17176b4448ad9cfbe5e1e7c33e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } }
I20260924 00:56:05.496120 10823 raft_consensus.cc:348] T 53cfcbe0257d44a283fcfc7b3fa42d88 P e51782766430416cba83a564995c41ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } }
I20260924 00:56:05.496316 10822 raft_consensus.cc:374] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 70748f17176b4448ad9cfbe5e1e7c33e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:05.496340 10823 raft_consensus.cc:374] T 53cfcbe0257d44a283fcfc7b3fa42d88 P e51782766430416cba83a564995c41ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:05.496395 10822 raft_consensus.cc:729] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 70748f17176b4448ad9cfbe5e1e7c33e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70748f17176b4448ad9cfbe5e1e7c33e, State: Initialized, Role: FOLLOWER
I20260924 00:56:05.496404 10823 raft_consensus.cc:729] T 53cfcbe0257d44a283fcfc7b3fa42d88 P e51782766430416cba83a564995c41ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51782766430416cba83a564995c41ba, State: Initialized, Role: FOLLOWER
I20260924 00:56:05.496533 10824 ts_tablet_manager.cc:1428] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 08a0bd05c3b04ce594713e51cec5e82a: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260924 00:56:05.496616 10823 consensus_queue.cc:260] T 53cfcbe0257d44a283fcfc7b3fa42d88 P e51782766430416cba83a564995c41ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } }
I20260924 00:56:05.496663 10822 consensus_queue.cc:260] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 70748f17176b4448ad9cfbe5e1e7c33e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } }
I20260924 00:56:05.497239 10822 ts_tablet_manager.cc:1428] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 70748f17176b4448ad9cfbe5e1e7c33e: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260924 00:56:05.497679 10823 ts_tablet_manager.cc:1428] T 53cfcbe0257d44a283fcfc7b3fa42d88 P e51782766430416cba83a564995c41ba: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260924 00:56:05.649161 10797 raft_consensus.cc:482] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 70748f17176b4448ad9cfbe5e1e7c33e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:56:05.649348 10797 raft_consensus.cc:504] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 70748f17176b4448ad9cfbe5e1e7c33e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } }
I20260924 00:56:05.649995 10797 leader_election.cc:288] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 70748f17176b4448ad9cfbe5e1e7c33e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e51782766430416cba83a564995c41ba (127.8.64.130:45425), 08a0bd05c3b04ce594713e51cec5e82a (127.8.64.131:35983)
I20260924 00:56:05.650633 10512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53cfcbe0257d44a283fcfc7b3fa42d88" candidate_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51782766430416cba83a564995c41ba" is_pre_election: true
I20260924 00:56:05.650692 10647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53cfcbe0257d44a283fcfc7b3fa42d88" candidate_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08a0bd05c3b04ce594713e51cec5e82a" is_pre_election: true
I20260924 00:56:05.650880 10512 raft_consensus.cc:2454] T 53cfcbe0257d44a283fcfc7b3fa42d88 P e51782766430416cba83a564995c41ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70748f17176b4448ad9cfbe5e1e7c33e in term 0.
I20260924 00:56:05.650909 10647 raft_consensus.cc:2454] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 08a0bd05c3b04ce594713e51cec5e82a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70748f17176b4448ad9cfbe5e1e7c33e in term 0.
I20260924 00:56:05.651325 10299 leader_election.cc:302] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 70748f17176b4448ad9cfbe5e1e7c33e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08a0bd05c3b04ce594713e51cec5e82a, 70748f17176b4448ad9cfbe5e1e7c33e; no voters:
I20260924 00:56:05.651583 10797 raft_consensus.cc:2781] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 70748f17176b4448ad9cfbe5e1e7c33e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:56:05.651690 10797 raft_consensus.cc:482] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 70748f17176b4448ad9cfbe5e1e7c33e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:56:05.651819 10797 raft_consensus.cc:3037] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 70748f17176b4448ad9cfbe5e1e7c33e [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:05.653927 10797 raft_consensus.cc:504] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 70748f17176b4448ad9cfbe5e1e7c33e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } }
I20260924 00:56:05.654584 10797 leader_election.cc:288] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 70748f17176b4448ad9cfbe5e1e7c33e [CANDIDATE]: Term 1 election: Requested vote from peers e51782766430416cba83a564995c41ba (127.8.64.130:45425), 08a0bd05c3b04ce594713e51cec5e82a (127.8.64.131:35983)
I20260924 00:56:05.654953 10512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53cfcbe0257d44a283fcfc7b3fa42d88" candidate_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51782766430416cba83a564995c41ba"
I20260924 00:56:05.655534 10512 raft_consensus.cc:3037] T 53cfcbe0257d44a283fcfc7b3fa42d88 P e51782766430416cba83a564995c41ba [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:05.655521 10647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53cfcbe0257d44a283fcfc7b3fa42d88" candidate_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08a0bd05c3b04ce594713e51cec5e82a"
I20260924 00:56:05.655743 10647 raft_consensus.cc:3037] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 08a0bd05c3b04ce594713e51cec5e82a [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:05.658517 10647 raft_consensus.cc:2454] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 08a0bd05c3b04ce594713e51cec5e82a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70748f17176b4448ad9cfbe5e1e7c33e in term 1.
I20260924 00:56:05.658509 10512 raft_consensus.cc:2454] T 53cfcbe0257d44a283fcfc7b3fa42d88 P e51782766430416cba83a564995c41ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70748f17176b4448ad9cfbe5e1e7c33e in term 1.
I20260924 00:56:05.658886 10302 leader_election.cc:302] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 70748f17176b4448ad9cfbe5e1e7c33e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70748f17176b4448ad9cfbe5e1e7c33e, e51782766430416cba83a564995c41ba; no voters:
I20260924 00:56:05.659217 10797 raft_consensus.cc:2781] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 70748f17176b4448ad9cfbe5e1e7c33e [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:56:05.659363 10797 raft_consensus.cc:686] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 70748f17176b4448ad9cfbe5e1e7c33e [term 1 LEADER]: Becoming Leader. State: Replica: 70748f17176b4448ad9cfbe5e1e7c33e, State: Running, Role: LEADER
I20260924 00:56:05.659741 10797 consensus_queue.cc:237] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 70748f17176b4448ad9cfbe5e1e7c33e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } }
I20260924 00:56:05.662262 10210 catalog_manager.cc:5495] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 70748f17176b4448ad9cfbe5e1e7c33e reported cstate change: term changed from 0 to 1, leader changed from <none> to 70748f17176b4448ad9cfbe5e1e7c33e (127.8.64.129). New cstate: current_term: 1 leader_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70748f17176b4448ad9cfbe5e1e7c33e" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 46381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 } health_report { overall_health: UNKNOWN } } }
I20260924 00:56:05.696085 10842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:05.686663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35698 (local address 127.8.64.129:46381)
0924 00:56:05.690686 (+ 4023us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:05.690690 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:05.690707 (+ 17us) server_negotiation.cc:407] Connection header received
0924 00:56:05.690755 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:05.690758 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:05.690815 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:05.690893 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:56:05.691583 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:05.692355 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:05.694342 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:05.694769 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:05.694866 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:05.695387 (+ 521us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:56:05.695493 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:56:05.695561 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:05.695690 (+ 129us) server_negotiation.cc:299] Negotiation successful
0924 00:56:05.695747 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3918,"thread_start_us":3814,"threads_started":1}
I20260924 00:56:05.709193 10842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:05.704464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:33651 (local address 127.8.64.129:46381)
0924 00:56:05.704664 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:05.704670 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:05.704826 (+ 156us) server_negotiation.cc:407] Connection header received
0924 00:56:05.705002 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:05.705007 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:05.705073 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:05.705160 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:56:05.705588 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:05.706231 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:05.707603 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:05.707998 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:05.708137 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:05.708476 (+ 339us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:56:05.708580 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:56:05.708670 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:05.708790 (+ 120us) server_negotiation.cc:299] Negotiation successful
0924 00:56:05.708904 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":56}
I20260924 00:56:05.709318 10845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:05.704167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:46381 (local address 127.8.64.129:33651)
0924 00:56:05.704737 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:05.704759 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:05.704891 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:05.705169 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:05.705173 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:05.705183 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 00:56:05.705434 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:05.705441 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:05.706400 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:05.706406 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:05.707468 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:05.707479 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:05.708150 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:05.708155 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:05.708304 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:05.708333 (+ 29us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 00:56:05.708670 (+ 337us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 00:56:05.708681 (+ 11us) client_negotiation.cc:769] Sending connection context
0924 00:56:05.708774 (+ 93us) client_negotiation.cc:240] Negotiation successful
0924 00:56:05.708963 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"mutex_wait_us":26,"thread_start_us":302,"threads_started":1}
I20260924 00:56:05.716665 10647 raft_consensus.cc:1261] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 08a0bd05c3b04ce594713e51cec5e82a [term 1 FOLLOWER]: Refusing update from remote peer 70748f17176b4448ad9cfbe5e1e7c33e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:56:05.716712 10512 raft_consensus.cc:1261] T 53cfcbe0257d44a283fcfc7b3fa42d88 P e51782766430416cba83a564995c41ba [term 1 FOLLOWER]: Refusing update from remote peer 70748f17176b4448ad9cfbe5e1e7c33e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:56:05.717208 10797 consensus_queue.cc:1035] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 70748f17176b4448ad9cfbe5e1e7c33e [LEADER]: Connected to new peer: Peer: permanent_uuid: "08a0bd05c3b04ce594713e51cec5e82a" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:56:05.717533 10724 consensus_queue.cc:1035] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 70748f17176b4448ad9cfbe5e1e7c33e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51782766430416cba83a564995c41ba" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260924 00:56:05.735999 10789 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260924 00:56:07.319087 10208 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51502:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:56:07.319325 10208 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51502:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:56:07.323019 10208 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d0cc323515b340cbb1ea1249b47797e8: Sending DeleteTablet for 3 replicas of tablet 53cfcbe0257d44a283fcfc7b3fa42d88
I20260924 00:56:07.323684 10488 tablet_service.cc:1514] Processing DeleteTablet for tablet 53cfcbe0257d44a283fcfc7b3fa42d88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:56:07 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:34520
I20260924 00:56:07.323681 10349 tablet_service.cc:1514] Processing DeleteTablet for tablet 53cfcbe0257d44a283fcfc7b3fa42d88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:56:07 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:43662
I20260924 00:56:07.323834 10627 tablet_service.cc:1514] Processing DeleteTablet for tablet 53cfcbe0257d44a283fcfc7b3fa42d88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:56:07 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:60822
I20260924 00:56:07.324575 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 10283
I20260924 00:56:07.337675 10868 tablet_replica.cc:330] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 70748f17176b4448ad9cfbe5e1e7c33e: stopping tablet replica
I20260924 00:56:07.337905 10868 raft_consensus.cc:2229] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 70748f17176b4448ad9cfbe5e1e7c33e [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:56:07.338505 10868 raft_consensus.cc:2258] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 70748f17176b4448ad9cfbe5e1e7c33e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:56:07.339802 10868 ts_tablet_manager.cc:1905] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 70748f17176b4448ad9cfbe5e1e7c33e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:56:07.341939 10869 tablet_replica.cc:330] T 53cfcbe0257d44a283fcfc7b3fa42d88 P e51782766430416cba83a564995c41ba: stopping tablet replica
I20260924 00:56:07.342960 10870 tablet_replica.cc:330] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 08a0bd05c3b04ce594713e51cec5e82a: stopping tablet replica
I20260924 00:56:07.343118 10869 raft_consensus.cc:2229] T 53cfcbe0257d44a283fcfc7b3fa42d88 P e51782766430416cba83a564995c41ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:56:07.343446 10869 raft_consensus.cc:2258] T 53cfcbe0257d44a283fcfc7b3fa42d88 P e51782766430416cba83a564995c41ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:56:07.343217 10870 raft_consensus.cc:2229] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 08a0bd05c3b04ce594713e51cec5e82a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:56:07.344007 10870 raft_consensus.cc:2258] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 08a0bd05c3b04ce594713e51cec5e82a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:56:07.344887 10869 ts_tablet_manager.cc:1905] T 53cfcbe0257d44a283fcfc7b3fa42d88 P e51782766430416cba83a564995c41ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:56:07.345486 10870 ts_tablet_manager.cc:1905] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 08a0bd05c3b04ce594713e51cec5e82a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:56:07.347896 10868 ts_tablet_manager.cc:1918] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 70748f17176b4448ad9cfbe5e1e7c33e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260924 00:56:07.348047 10868 log.cc:1196] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 70748f17176b4448ad9cfbe5e1e7c33e: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-0/wal/wals/53cfcbe0257d44a283fcfc7b3fa42d88
I20260924 00:56:07.348506 10868 ts_tablet_manager.cc:1939] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 70748f17176b4448ad9cfbe5e1e7c33e: Deleting consensus metadata
I20260924 00:56:07.349597 10197 catalog_manager.cc:4841] TS 70748f17176b4448ad9cfbe5e1e7c33e (127.8.64.129:46381): tablet 53cfcbe0257d44a283fcfc7b3fa42d88 (table test-table [id=a7e1137a0e204f5ea4580995e32f467f]) successfully deleted
I20260924 00:56:07.351444 10869 ts_tablet_manager.cc:1918] T 53cfcbe0257d44a283fcfc7b3fa42d88 P e51782766430416cba83a564995c41ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260924 00:56:07.351580 10869 log.cc:1196] T 53cfcbe0257d44a283fcfc7b3fa42d88 P e51782766430416cba83a564995c41ba: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-1/wal/wals/53cfcbe0257d44a283fcfc7b3fa42d88
I20260924 00:56:07.352011 10869 ts_tablet_manager.cc:1939] T 53cfcbe0257d44a283fcfc7b3fa42d88 P e51782766430416cba83a564995c41ba: Deleting consensus metadata
I20260924 00:56:07.352294 10870 ts_tablet_manager.cc:1918] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 08a0bd05c3b04ce594713e51cec5e82a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260924 00:56:07.352490 10870 log.cc:1196] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 08a0bd05c3b04ce594713e51cec5e82a: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790211343099518-8450-0/minicluster-data/ts-2/wal/wals/53cfcbe0257d44a283fcfc7b3fa42d88
I20260924 00:56:07.352950 10870 ts_tablet_manager.cc:1939] T 53cfcbe0257d44a283fcfc7b3fa42d88 P 08a0bd05c3b04ce594713e51cec5e82a: Deleting consensus metadata
I20260924 00:56:07.353200 10198 catalog_manager.cc:4841] TS e51782766430416cba83a564995c41ba (127.8.64.130:45425): tablet 53cfcbe0257d44a283fcfc7b3fa42d88 (table test-table [id=a7e1137a0e204f5ea4580995e32f467f]) successfully deleted
I20260924 00:56:07.353859 10195 catalog_manager.cc:4841] TS 08a0bd05c3b04ce594713e51cec5e82a (127.8.64.131:35983): tablet 53cfcbe0257d44a283fcfc7b3fa42d88 (table test-table [id=a7e1137a0e204f5ea4580995e32f467f]) successfully deleted
Sep 24 00:56:07 dist-test-slave-mmhs krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211365, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.64.129@KRBTEST.COM
I20260924 00:56:07.364429 10871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:07.325463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35702 (local address 127.8.64.129:46381)
0924 00:56:07.343612 (+ 18149us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:07.343618 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:07.343645 (+ 27us) server_negotiation.cc:407] Connection header received
0924 00:56:07.343704 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:07.343710 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:07.343783 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:07.345639 (+ 1856us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:07.345828 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:07.347121 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:07.349075 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:07.349699 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:07.350395 (+ 696us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:07.357895 (+ 7500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:07.357910 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:07.357925 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:07.357959 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:07.361515 (+ 3556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:07.362192 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:07.362208 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:07.362210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:07.362421 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:07.362818 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:07.362824 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:07.362827 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:07.363181 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:07.363448 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:07.363844 (+ 396us) server_negotiation.cc:299] Negotiation successful
0924 00:56:07.364077 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18011,"thread_start_us":17904,"threads_started":1}
W20260924 00:56:07.365180 10415 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35702
W20260924 00:56:07.365833 8450 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 10283: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260924 00:56:07.366078 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 10283
I20260924 00:56:07.393752 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 10424
Sep 24 00:56:07 dist-test-slave-mmhs krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211365, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.64.130@KRBTEST.COM
I20260924 00:56:07.408183 10873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:07.394537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48138 (local address 127.8.64.130:45425)
0924 00:56:07.395072 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:07.395076 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:07.395093 (+ 17us) server_negotiation.cc:407] Connection header received
0924 00:56:07.395153 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:07.395157 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:07.395215 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:07.395305 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:07.396465 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:07.397283 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:07.398566 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:07.399073 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:07.399189 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:07.403070 (+ 3881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:07.403084 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:07.403101 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:07.403159 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:07.405822 (+ 2663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:07.406450 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:07.406457 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:07.406460 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:07.406586 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:07.406901 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:07.406904 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:07.406906 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:07.407237 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:07.407457 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:07.407645 (+ 188us) server_negotiation.cc:299] Negotiation successful
0924 00:56:07.407801 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":258,"threads_started":1}
W20260924 00:56:07.408838 10554 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48138
W20260924 00:56:07.409510 8450 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 10424: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260924 00:56:07.409675 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 10424
I20260924 00:56:07.432245 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 10563
Sep 24 00:56:07 dist-test-slave-mmhs krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211365, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.64.131@KRBTEST.COM
I20260924 00:56:07.446705 10874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:07.432992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48864 (local address 127.8.64.131:35983)
0924 00:56:07.433564 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:07.433568 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:07.433584 (+ 16us) server_negotiation.cc:407] Connection header received
0924 00:56:07.433642 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:07.433646 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:07.433702 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:07.433796 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:07.435126 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:07.435893 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:07.437087 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:07.437514 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:07.437608 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:07.441595 (+ 3987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:07.441610 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:07.441626 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:07.441659 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:07.444259 (+ 2600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:07.444865 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:07.444872 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:07.444874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:07.444998 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:07.445322 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:07.445326 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:07.445328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:07.445665 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:07.445889 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:07.446145 (+ 256us) server_negotiation.cc:299] Negotiation successful
0924 00:56:07.446426 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":342,"threads_started":1}
W20260924 00:56:07.447314 10693 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48864
W20260924 00:56:07.447896 8450 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 10563: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260924 00:56:07.448037 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 10563
I20260924 00:56:07.470271 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 10179
I20260924 00:56:07.471722 10251 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:56:07.584991 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 10179
2026-09-24T00:56:07Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8711 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:56:07 dist-test-slave-mmhs krb5kdc[10878](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:56:07 dist-test-slave-mmhs krb5kdc[10878](info): set up 2 sockets
Sep 24 00:56:07 dist-test-slave-mmhs krb5kdc[10878](info): commencing operation
krb5kdc: starting...
W20260924 00:56:08.688544 8450 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.000s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Sep 24 00:56:08 dist-test-slave-mmhs krb5kdc[10878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:56:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:56:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.64.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
I20260924 00:56:08.879118 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:40237
--webserver_interface=127.8.64.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:32939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:40237
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:56:09.188781 10892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:09.189265 10892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:09.189419 10892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:09.199846 10892 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:56:09.199990 10892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:09.200068 10892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:09.200160 10892 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:56:09.200224 10892 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:56:09.209977 10892 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:32939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:40237
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:40237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:09.211321 10892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:09.212601 10892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:09.224618 10897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:09.224632 10898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:09.224789 10892 server_base.cc:1034] running on GCE node
W20260924 00:56:09.224560 10900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:09.225787 10892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:09.227239 10892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:09.228435 10892 hybrid_clock.cc:648] HybridClock initialized: now 1790211369228389 us; error 45 us; skew 500 ppm
I20260924 00:56:09.228770 10892 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:09 dist-test-slave-mmhs krb5kdc[10878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211369, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:09.231775 10892 init.cc:377] Logged in from keytab as kudu/127.8.64.190@KRBTEST.COM (short username kudu)
I20260924 00:56:09.233228 10892 webserver.cc:460] Webserver started at http://127.8.64.190:33057/ using document root <none> and password file <none>
I20260924 00:56:09.233843 10892 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:09.233944 10892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:09.234192 10892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:56:09.237035 10892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "46d0e83d25a84bf3b652caf5012d192b"
format_stamp: "Formatted at 2026-09-24 00:56:09 on dist-test-slave-mmhs"
server_key: "df24ab552e3d5dd89eef10a9da0d9405"
server_key_iv: "49729ab675381da28c2abeb3469fbffe"
server_key_version: "encryptionkey@0"
I20260924 00:56:09.237740 10892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "46d0e83d25a84bf3b652caf5012d192b"
format_stamp: "Formatted at 2026-09-24 00:56:09 on dist-test-slave-mmhs"
server_key: "df24ab552e3d5dd89eef10a9da0d9405"
server_key_iv: "49729ab675381da28c2abeb3469fbffe"
server_key_version: "encryptionkey@0"
I20260924 00:56:09.242647 10892 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.006s
I20260924 00:56:09.245836 10907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:09.247187 10892 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 00:56:09.247324 10892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "46d0e83d25a84bf3b652caf5012d192b"
format_stamp: "Formatted at 2026-09-24 00:56:09 on dist-test-slave-mmhs"
server_key: "df24ab552e3d5dd89eef10a9da0d9405"
server_key_iv: "49729ab675381da28c2abeb3469fbffe"
server_key_version: "encryptionkey@0"
I20260924 00:56:09.247479 10892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:09.277241 10892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:09.281073 10892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:09.281312 10892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:09.299208 10892 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.190:40237
I20260924 00:56:09.299242 10958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.190:40237 every 8 connection(s)
I20260924 00:56:09.300518 10892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 00:56:09.304226 10959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:09.306165 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 10892
I20260924 00:56:09.306420 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 00:56:09.307214 8450 external_mini_cluster.cc:1428] Setting key f50e817f041777f2b4c53a83f027be2f
I20260924 00:56:09.315900 10959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46d0e83d25a84bf3b652caf5012d192b: Bootstrap starting.
I20260924 00:56:09.318903 10959 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46d0e83d25a84bf3b652caf5012d192b: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:09.319846 10959 log.cc:824] T 00000000000000000000000000000000 P 46d0e83d25a84bf3b652caf5012d192b: Log is configured to *not* fsync() on all Append() calls
Sep 24 00:56:09 dist-test-slave-mmhs krb5kdc[10878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:56:09.323051 10959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46d0e83d25a84bf3b652caf5012d192b: No bootstrap required, opened a new log
I20260924 00:56:09.329819 10962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:09.309237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50990 (local address 127.8.64.190:40237)
0924 00:56:09.309988 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:09.309999 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:09.310037 (+ 38us) server_negotiation.cc:407] Connection header received
0924 00:56:09.311712 (+ 1675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:09.311735 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:09.312111 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:09.312481 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:09.313936 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:09.315299 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:09.316642 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:09.317298 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:09.317490 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:09.321659 (+ 4169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:09.321669 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:09.321686 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:09.321725 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:09.325578 (+ 3853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:09.326191 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:09.326199 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:09.326206 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:09.326441 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:09.326782 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:09.326788 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:09.326791 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:09.327502 (+ 711us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:09.327771 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:09.328095 (+ 324us) server_negotiation.cc:299] Negotiation successful
0924 00:56:09.328532 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":315,"threads_started":1}
I20260924 00:56:09.333295 10959 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46d0e83d25a84bf3b652caf5012d192b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46d0e83d25a84bf3b652caf5012d192b" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 40237 } }
I20260924 00:56:09.333595 10959 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46d0e83d25a84bf3b652caf5012d192b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:09.333671 10959 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46d0e83d25a84bf3b652caf5012d192b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46d0e83d25a84bf3b652caf5012d192b, State: Initialized, Role: FOLLOWER
I20260924 00:56:09.334160 10959 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46d0e83d25a84bf3b652caf5012d192b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46d0e83d25a84bf3b652caf5012d192b" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 40237 } }
I20260924 00:56:09.334398 10959 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46d0e83d25a84bf3b652caf5012d192b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:56:09.334497 10959 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46d0e83d25a84bf3b652caf5012d192b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:56:09.334625 10959 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46d0e83d25a84bf3b652caf5012d192b [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:09.336970 10959 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46d0e83d25a84bf3b652caf5012d192b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46d0e83d25a84bf3b652caf5012d192b" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 40237 } }
I20260924 00:56:09.337387 10959 leader_election.cc:302] T 00000000000000000000000000000000 P 46d0e83d25a84bf3b652caf5012d192b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46d0e83d25a84bf3b652caf5012d192b; no voters:
I20260924 00:56:09.337958 10959 leader_election.cc:288] T 00000000000000000000000000000000 P 46d0e83d25a84bf3b652caf5012d192b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:56:09.338068 10964 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46d0e83d25a84bf3b652caf5012d192b [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:56:09.338794 10964 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46d0e83d25a84bf3b652caf5012d192b [term 1 LEADER]: Becoming Leader. State: Replica: 46d0e83d25a84bf3b652caf5012d192b, State: Running, Role: LEADER
I20260924 00:56:09.339282 10964 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46d0e83d25a84bf3b652caf5012d192b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46d0e83d25a84bf3b652caf5012d192b" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 40237 } }
I20260924 00:56:09.339946 10959 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46d0e83d25a84bf3b652caf5012d192b [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:56:09.341852 10965 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46d0e83d25a84bf3b652caf5012d192b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46d0e83d25a84bf3b652caf5012d192b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46d0e83d25a84bf3b652caf5012d192b" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 40237 } } }
I20260924 00:56:09.342249 10965 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46d0e83d25a84bf3b652caf5012d192b [sys.catalog]: This master's current role is: LEADER
I20260924 00:56:09.342149 10966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46d0e83d25a84bf3b652caf5012d192b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46d0e83d25a84bf3b652caf5012d192b. Latest consensus state: current_term: 1 leader_uuid: "46d0e83d25a84bf3b652caf5012d192b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46d0e83d25a84bf3b652caf5012d192b" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 40237 } } }
I20260924 00:56:09.342751 10966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46d0e83d25a84bf3b652caf5012d192b [sys.catalog]: This master's current role is: LEADER
I20260924 00:56:09.344200 10971 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:56:09.349308 10971 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:56:09.358265 10971 catalog_manager.cc:1347] Generated new cluster ID: e21b094f04184e9e9f3abaa53002a553
I20260924 00:56:09.358474 10971 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:56:09.378002 10971 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:56:09.379909 10971 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:56:09.388649 10971 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46d0e83d25a84bf3b652caf5012d192b: Generated new TSK 0
I20260924 00:56:09.389626 10971 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.64.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab.
Entry for principal kudu/127.8.64.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab.
I20260924 00:56:09.446612 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.129:0
--local_ip_for_outbound_sockets=127.8.64.129
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:40237
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab
--principal=kudu/127.8.64.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:32939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:56:09.767397 10985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:09.767658 10985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:09.767748 10985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:09.774791 10985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:09.774904 10985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:09.775013 10985 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.129
I20260924 00:56:09.784454 10985 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:32939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab
--principal=kudu/127.8.64.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:40237
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.129
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:09.785640 10985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:09.786922 10985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:09.799892 10990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:09.799856 10993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:09.799924 10991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:09.800740 10985 server_base.cc:1034] running on GCE node
I20260924 00:56:09.801345 10985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:09.802255 10985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:09.803503 10985 hybrid_clock.cc:648] HybridClock initialized: now 1790211369803445 us; error 49 us; skew 500 ppm
I20260924 00:56:09.803812 10985 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:09 dist-test-slave-mmhs krb5kdc[10878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211369, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:09.807056 10985 init.cc:377] Logged in from keytab as kudu/127.8.64.129@KRBTEST.COM (short username kudu)
I20260924 00:56:09.808411 10985 webserver.cc:460] Webserver started at http://127.8.64.129:38119/ using document root <none> and password file <none>
I20260924 00:56:09.809070 10985 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:09.809206 10985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:09.809510 10985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:56:09.812362 10985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-0/data/instance:
uuid: "fe2c2797061649079ba7be3a45fb8851"
format_stamp: "Formatted at 2026-09-24 00:56:09 on dist-test-slave-mmhs"
server_key: "e636c4e5751da26b34147d9a5356a520"
server_key_iv: "6bd3875f676ccf5fbcf78dec03faa1ad"
server_key_version: "encryptionkey@0"
I20260924 00:56:09.813129 10985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance:
uuid: "fe2c2797061649079ba7be3a45fb8851"
format_stamp: "Formatted at 2026-09-24 00:56:09 on dist-test-slave-mmhs"
server_key: "e636c4e5751da26b34147d9a5356a520"
server_key_iv: "6bd3875f676ccf5fbcf78dec03faa1ad"
server_key_version: "encryptionkey@0"
I20260924 00:56:09.818020 10985 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260924 00:56:09.821292 11000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:09.822527 10985 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 00:56:09.822691 10985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-0/wal
uuid: "fe2c2797061649079ba7be3a45fb8851"
format_stamp: "Formatted at 2026-09-24 00:56:09 on dist-test-slave-mmhs"
server_key: "e636c4e5751da26b34147d9a5356a520"
server_key_iv: "6bd3875f676ccf5fbcf78dec03faa1ad"
server_key_version: "encryptionkey@0"
I20260924 00:56:09.822852 10985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:09.842638 10985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:09.846947 10985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:09.847230 10985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:09.848182 10985 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:56:09.849443 10985 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:56:09.849515 10985 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:09.849682 10985 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:56:09.849745 10985 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:09.880134 10985 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.129:32941
I20260924 00:56:09.880179 11113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.129:32941 every 8 connection(s)
I20260924 00:56:09.881398 10985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
I20260924 00:56:09.884704 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 10985
I20260924 00:56:09.884933 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance
I20260924 00:56:09.885690 8450 external_mini_cluster.cc:1428] Setting key cc1ceecf5f3788411e3e57b0797c8f0a
Sep 24 00:56:09 dist-test-slave-mmhs krb5kdc[10878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211369, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:56:09.902346 11116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:09.883827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:46913 (local address 127.8.64.190:40237)
0924 00:56:09.884296 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:09.884301 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:09.885306 (+ 1005us) server_negotiation.cc:407] Connection header received
0924 00:56:09.887485 (+ 2179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:09.887489 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:09.887574 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:09.887711 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:09.889364 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:09.890527 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:09.892115 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:09.892830 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:09.892973 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:09.896371 (+ 3398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:09.896376 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:09.896379 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:09.896408 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:09.899142 (+ 2734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:09.900036 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:09.900044 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:09.900050 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:09.900185 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:09.900654 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:09.900659 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:09.900662 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:09.900913 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:09.901065 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:09.901742 (+ 677us) server_negotiation.cc:299] Negotiation successful
0924 00:56:09.901950 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":270,"threads_started":1}
I20260924 00:56:09.903373 11117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:09.884291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:40237 (local address 127.8.64.129:46913)
0924 00:56:09.885153 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:09.885193 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:09.887176 (+ 1983us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:09.888060 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:09.888071 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:09.888640 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:09.889195 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:09.889213 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:09.890667 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:09.890670 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:09.891968 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:09.891980 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:09.893061 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:09.893067 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:09.893264 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:09.893958 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:09.893983 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:09.896161 (+ 2178us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:09.899281 (+ 3120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:09.899318 (+ 37us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:09.899337 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:09.899920 (+ 583us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:09.900293 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:09.900298 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:09.900302 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:09.900539 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:09.901038 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:09.901046 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:09.901386 (+ 340us) client_negotiation.cc:769] Sending connection context
0924 00:56:09.901662 (+ 276us) client_negotiation.cc:240] Negotiation successful
0924 00:56:09.902173 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":327,"threads_started":1}
WARNING: no policy specified for kudu/127.8.64.130@KRBTEST.COM; defaulting to no policy
I20260924 00:56:09.904982 11114 heartbeater.cc:347] Connected to a master server at 127.8.64.190:40237
I20260924 00:56:09.905342 11114 heartbeater.cc:464] Registering TS with master...
I20260924 00:56:09.906041 11114 heartbeater.cc:511] Master 127.8.64.190:40237 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.130@KRBTEST.COM" created.
I20260924 00:56:09.908437 10924 ts_manager.cc:194] Registered new tserver with Master: fe2c2797061649079ba7be3a45fb8851 (127.8.64.129:32941)
I20260924 00:56:09.910539 10924 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.129@KRBTEST.COM'} at 127.8.64.129:46913
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab.
Entry for principal kudu/127.8.64.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab.
I20260924 00:56:09.930051 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.130:0
--local_ip_for_outbound_sockets=127.8.64.130
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:40237
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab
--principal=kudu/127.8.64.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:32939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:56:10.255174 11120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:10.255465 11120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:10.255568 11120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:10.262957 11120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:10.263079 11120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:10.263193 11120 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.130
I20260924 00:56:10.272795 11120 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:32939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab
--principal=kudu/127.8.64.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:40237
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.130
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:10.274075 11120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:10.275417 11120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:10.289007 11125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:10.288985 11128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:10.289176 11126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:10.289830 11120 server_base.cc:1034] running on GCE node
I20260924 00:56:10.290485 11120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:10.291411 11120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:10.292668 11120 hybrid_clock.cc:648] HybridClock initialized: now 1790211370292615 us; error 57 us; skew 500 ppm
I20260924 00:56:10.293020 11120 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:10 dist-test-slave-mmhs krb5kdc[10878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211370, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:10.296154 11120 init.cc:377] Logged in from keytab as kudu/127.8.64.130@KRBTEST.COM (short username kudu)
I20260924 00:56:10.297660 11120 webserver.cc:460] Webserver started at http://127.8.64.130:37939/ using document root <none> and password file <none>
I20260924 00:56:10.298429 11120 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:10.298570 11120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:10.298868 11120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:56:10.301676 11120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-1/data/instance:
uuid: "2183e63afa1c4504a7e0970914a397ae"
format_stamp: "Formatted at 2026-09-24 00:56:10 on dist-test-slave-mmhs"
server_key: "e3261f1c9b86f21a86a7e2f16ee42067"
server_key_iv: "be1fa75568d9aab3ce5e8360923cab51"
server_key_version: "encryptionkey@0"
I20260924 00:56:10.302531 11120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance:
uuid: "2183e63afa1c4504a7e0970914a397ae"
format_stamp: "Formatted at 2026-09-24 00:56:10 on dist-test-slave-mmhs"
server_key: "e3261f1c9b86f21a86a7e2f16ee42067"
server_key_iv: "be1fa75568d9aab3ce5e8360923cab51"
server_key_version: "encryptionkey@0"
I20260924 00:56:10.307670 11120 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260924 00:56:10.311116 11135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:10.312491 11120 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260924 00:56:10.312623 11120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-1/wal
uuid: "2183e63afa1c4504a7e0970914a397ae"
format_stamp: "Formatted at 2026-09-24 00:56:10 on dist-test-slave-mmhs"
server_key: "e3261f1c9b86f21a86a7e2f16ee42067"
server_key_iv: "be1fa75568d9aab3ce5e8360923cab51"
server_key_version: "encryptionkey@0"
I20260924 00:56:10.312775 11120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:10.333243 11120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:10.337262 11120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:10.337508 11120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:10.338446 11120 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:56:10.339676 11120 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:56:10.339766 11120 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:10.339852 11120 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:56:10.339896 11120 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:10.373041 11120 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.130:39985
I20260924 00:56:10.373131 11247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.130:39985 every 8 connection(s)
I20260924 00:56:10.374434 11120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
I20260924 00:56:10.378087 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 11120
I20260924 00:56:10.378340 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance
I20260924 00:56:10.378872 8450 external_mini_cluster.cc:1428] Setting key c90c3536b1acd830ac8dc8db44ce0a4d
Sep 24 00:56:10 dist-test-slave-mmhs krb5kdc[10878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211370, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.130@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:56:10.397159 11116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:10.377089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:60497 (local address 127.8.64.190:40237)
0924 00:56:10.377289 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:10.377295 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:10.378590 (+ 1295us) server_negotiation.cc:407] Connection header received
0924 00:56:10.380640 (+ 2050us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:10.380645 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:10.380731 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:10.380850 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:10.382325 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:10.383180 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:10.385784 (+ 2604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:10.386550 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:10.386696 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:10.390008 (+ 3312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:10.390017 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:10.390025 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:10.390069 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:10.393650 (+ 3581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:10.394482 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:10.394490 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:10.394497 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:10.394669 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:10.395225 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:10.395231 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:10.395235 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:10.395647 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:10.395817 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:10.396498 (+ 681us) server_negotiation.cc:299] Negotiation successful
0924 00:56:10.396756 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260924 00:56:10.397887 11250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:10.377559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:40237 (local address 127.8.64.130:60497)
0924 00:56:10.378421 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:10.378466 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:10.380346 (+ 1880us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:10.381080 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:10.381087 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:10.381587 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:10.382132 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:10.382148 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:10.383381 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:10.383389 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:10.385614 (+ 2225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:10.385632 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:10.386763 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:10.386768 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:10.386960 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:10.387696 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:10.387722 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:10.389789 (+ 2067us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:10.393799 (+ 4010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:10.393806 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:10.393821 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:10.394251 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:10.394819 (+ 568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:10.394827 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:10.394832 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:10.395092 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:10.395793 (+ 701us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:10.395798 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:10.396120 (+ 322us) client_negotiation.cc:769] Sending connection context
0924 00:56:10.396363 (+ 243us) client_negotiation.cc:240] Negotiation successful
0924 00:56:10.396754 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":344,"threads_started":1}
WARNING: no policy specified for kudu/127.8.64.131@KRBTEST.COM; defaulting to no policy
I20260924 00:56:10.399523 11248 heartbeater.cc:347] Connected to a master server at 127.8.64.190:40237
I20260924 00:56:10.399835 11248 heartbeater.cc:464] Registering TS with master...
I20260924 00:56:10.400627 11248 heartbeater.cc:511] Master 127.8.64.190:40237 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.131@KRBTEST.COM" created.
I20260924 00:56:10.402186 10924 ts_manager.cc:194] Registered new tserver with Master: 2183e63afa1c4504a7e0970914a397ae (127.8.64.130:39985)
I20260924 00:56:10.403015 10924 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.130@KRBTEST.COM'} at 127.8.64.130:60497
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab.
Entry for principal kudu/127.8.64.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab.
I20260924 00:56:10.427129 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.131:0
--local_ip_for_outbound_sockets=127.8.64.131
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:40237
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab
--principal=kudu/127.8.64.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:32939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:56:10.742830 11253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:10.743108 11253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:10.743259 11253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:10.750680 11253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:10.750797 11253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:10.750972 11253 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.131
I20260924 00:56:10.760624 11253 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:32939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab
--principal=kudu/127.8.64.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:40237
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.131
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:10.761917 11253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:10.763255 11253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:10.776413 11259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:10.776453 11261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:10.776511 11258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:10.777616 11253 server_base.cc:1034] running on GCE node
I20260924 00:56:10.778188 11253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:10.779145 11253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:10.780337 11253 hybrid_clock.cc:648] HybridClock initialized: now 1790211370780279 us; error 54 us; skew 500 ppm
I20260924 00:56:10.780634 11253 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:10 dist-test-slave-mmhs krb5kdc[10878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211370, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:10.783754 11253 init.cc:377] Logged in from keytab as kudu/127.8.64.131@KRBTEST.COM (short username kudu)
I20260924 00:56:10.785137 11253 webserver.cc:460] Webserver started at http://127.8.64.131:39965/ using document root <none> and password file <none>
I20260924 00:56:10.785828 11253 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:10.785956 11253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:10.786283 11253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:56:10.789204 11253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-2/data/instance:
uuid: "20c28d2523f34bc99aa951438cbf735a"
format_stamp: "Formatted at 2026-09-24 00:56:10 on dist-test-slave-mmhs"
server_key: "e3f01d8ad0f307ab4b121a0b02aeab50"
server_key_iv: "2e6c82fa28bd1edfcf0bdd7d6669f4cc"
server_key_version: "encryptionkey@0"
I20260924 00:56:10.789963 11253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance:
uuid: "20c28d2523f34bc99aa951438cbf735a"
format_stamp: "Formatted at 2026-09-24 00:56:10 on dist-test-slave-mmhs"
server_key: "e3f01d8ad0f307ab4b121a0b02aeab50"
server_key_iv: "2e6c82fa28bd1edfcf0bdd7d6669f4cc"
server_key_version: "encryptionkey@0"
I20260924 00:56:10.794906 11253 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260924 00:56:10.798174 11268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:10.799472 11253 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260924 00:56:10.799642 11253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-2/wal
uuid: "20c28d2523f34bc99aa951438cbf735a"
format_stamp: "Formatted at 2026-09-24 00:56:10 on dist-test-slave-mmhs"
server_key: "e3f01d8ad0f307ab4b121a0b02aeab50"
server_key_iv: "2e6c82fa28bd1edfcf0bdd7d6669f4cc"
server_key_version: "encryptionkey@0"
I20260924 00:56:10.799856 11253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:10.833451 11253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:10.837415 11253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:10.837708 11253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:10.838725 11253 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:56:10.840063 11253 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:56:10.840171 11253 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:10.840288 11253 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:56:10.840334 11253 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:10.873443 11253 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.131:39827
I20260924 00:56:10.873539 11382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.131:39827 every 8 connection(s)
I20260924 00:56:10.874809 11253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
I20260924 00:56:10.875833 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 11253
I20260924 00:56:10.876060 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance
I20260924 00:56:10.876535 8450 external_mini_cluster.cc:1428] Setting key c9da37a0fad92d81613830212884817a
Sep 24 00:56:10 dist-test-slave-mmhs krb5kdc[10878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211370, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.131@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:56:10.895924 11116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:10.877551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:54053 (local address 127.8.64.190:40237)
0924 00:56:10.877759 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:10.877765 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:10.879236 (+ 1471us) server_negotiation.cc:407] Connection header received
0924 00:56:10.881755 (+ 2519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:10.881759 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:10.881848 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:10.881933 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:10.883927 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:10.884670 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:10.886497 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:10.886891 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:10.886987 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:10.890403 (+ 3416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:10.890408 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:10.890411 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:10.890440 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:10.893350 (+ 2910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:10.893998 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:10.894002 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:10.894004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:10.894140 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:10.894572 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:10.894575 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:10.894577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:10.894824 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:10.894931 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:10.895516 (+ 585us) server_negotiation.cc:299] Negotiation successful
0924 00:56:10.895680 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260924 00:56:10.896845 11385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:10.878233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:40237 (local address 127.8.64.131:54053)
0924 00:56:10.879065 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:10.879117 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:10.881488 (+ 2371us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:10.882192 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:10.882204 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:10.882933 (+ 729us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:10.883725 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:10.883747 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:10.884825 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:10.884830 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:10.886376 (+ 1546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:10.886388 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:10.887076 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:10.887081 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:10.887265 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:10.887914 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:10.887939 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:10.890170 (+ 2231us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:10.893475 (+ 3305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:10.893481 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:10.893492 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:10.893893 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:10.894228 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:10.894232 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:10.894234 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:10.894469 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:10.894926 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:10.894932 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:10.895214 (+ 282us) client_negotiation.cc:769] Sending connection context
0924 00:56:10.895422 (+ 208us) client_negotiation.cc:240] Negotiation successful
0924 00:56:10.895779 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":311,"threads_started":1}
I20260924 00:56:10.898140 11383 heartbeater.cc:347] Connected to a master server at 127.8.64.190:40237
I20260924 00:56:10.898463 11383 heartbeater.cc:464] Registering TS with master...
I20260924 00:56:10.899133 11383 heartbeater.cc:511] Master 127.8.64.190:40237 requested a full tablet report, sending...
I20260924 00:56:10.900456 10924 ts_manager.cc:194] Registered new tserver with Master: 20c28d2523f34bc99aa951438cbf735a (127.8.64.131:39827)
I20260924 00:56:10.901203 10924 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.131@KRBTEST.COM'} at 127.8.64.131:54053
I20260924 00:56:10.904132 8450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 00:56:10.913443 11114 heartbeater.cc:503] Master 127.8.64.190:40237 was elected leader, sending a full tablet report...
W20260924 00:56:10.934996 11116 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:56:10.932484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50998 (local address 127.8.64.190:40237)
0924 00:56:10.932721 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:10.932730 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:10.933116 (+ 386us) server_negotiation.cc:407] Connection header received
0924 00:56:10.933285 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:10.933289 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:10.933353 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:10.933441 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:10.934742 (+ 1301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50998: BlockingRecv error: recv got EOF from 127.0.0.1:50998 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260924 00:56:10.937737 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 10985
W20260924 00:56:10.940354 8450 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 10985: Not authorized: Client connection negotiation failed: client connection to 127.8.64.129:32941: server requires authentication, but client does not have Kerberos credentials available
I20260924 00:56:10.940551 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 10985
I20260924 00:56:10.956773 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 11120
W20260924 00:56:10.959533 8450 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 11120: Not authorized: Client connection negotiation failed: client connection to 127.8.64.130:39985: server requires authentication, but client does not have Kerberos credentials available
I20260924 00:56:10.959712 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 11120
I20260924 00:56:10.976142 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 11253
W20260924 00:56:10.978828 8450 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 11253: Not authorized: Client connection negotiation failed: client connection to 127.8.64.131:39827: server requires authentication, but client does not have Kerberos credentials available
I20260924 00:56:10.978971 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 11253
I20260924 00:56:10.994159 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 10892
I20260924 00:56:10.995424 10954 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:56:11.095798 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 10892
2026-09-24T00:56:11Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3507 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790211343099518-8450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:56:11 dist-test-slave-mmhs krb5kdc[11400](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:56:11 dist-test-slave-mmhs krb5kdc[11400](info): set up 2 sockets
Sep 24 00:56:11 dist-test-slave-mmhs krb5kdc[11400](info): commencing operation
krb5kdc: starting...
W20260924 00:56:12.180325 8450 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.000s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Sep 24 00:56:12 dist-test-slave-mmhs krb5kdc[11400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:56:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:56:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.64.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
I20260924 00:56:12.373562 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:43795
--webserver_interface=127.8.64.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:32935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:43795
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:56:12.693972 11413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:12.694279 11413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:12.694497 11413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:12.701797 11413 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:56:12.701915 11413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:12.702001 11413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:12.702121 11413 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:56:12.702215 11413 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:56:12.712363 11413 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:32935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:43795
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:43795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:12.713704 11413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:12.715062 11413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:12.726796 11418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:12.726766 11419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:12.726933 11421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:12.728104 11413 server_base.cc:1034] running on GCE node
I20260924 00:56:12.728921 11413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:12.730485 11413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:12.731760 11413 hybrid_clock.cc:648] HybridClock initialized: now 1790211372731689 us; error 55 us; skew 500 ppm
I20260924 00:56:12.732172 11413 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:12 dist-test-slave-mmhs krb5kdc[11400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211372, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:12.741999 11413 init.cc:377] Logged in from keytab as kudu/127.8.64.190@KRBTEST.COM (short username kudu)
I20260924 00:56:12.743947 11413 webserver.cc:460] Webserver started at http://127.8.64.190:36943/ using document root <none> and password file <none>
I20260924 00:56:12.744742 11413 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:12.744885 11413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:12.745246 11413 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:56:12.748569 11413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "c44cfd31e13747a9bcde8fc9f7bbb365"
format_stamp: "Formatted at 2026-09-24 00:56:12 on dist-test-slave-mmhs"
server_key: "dc911e41a5d574712cf5129fb41505ae"
server_key_iv: "9c34d98c7d4afd1c682cb66bcb3fc058"
server_key_version: "encryptionkey@0"
I20260924 00:56:12.749424 11413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "c44cfd31e13747a9bcde8fc9f7bbb365"
format_stamp: "Formatted at 2026-09-24 00:56:12 on dist-test-slave-mmhs"
server_key: "dc911e41a5d574712cf5129fb41505ae"
server_key_iv: "9c34d98c7d4afd1c682cb66bcb3fc058"
server_key_version: "encryptionkey@0"
I20260924 00:56:12.755131 11413 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260924 00:56:12.758836 11428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:12.760211 11413 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260924 00:56:12.760403 11413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "c44cfd31e13747a9bcde8fc9f7bbb365"
format_stamp: "Formatted at 2026-09-24 00:56:12 on dist-test-slave-mmhs"
server_key: "dc911e41a5d574712cf5129fb41505ae"
server_key_iv: "9c34d98c7d4afd1c682cb66bcb3fc058"
server_key_version: "encryptionkey@0"
I20260924 00:56:12.760613 11413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:12.781100 11413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:12.785413 11413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:12.785724 11413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:12.804307 11413 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.190:43795
I20260924 00:56:12.804338 11479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.190:43795 every 8 connection(s)
I20260924 00:56:12.805661 11413 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 00:56:12.809257 11480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:12.812364 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 11413
I20260924 00:56:12.812566 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 00:56:12.813345 8450 external_mini_cluster.cc:1428] Setting key f6bb346b8fff5e5b06df38b59e3f2f84
I20260924 00:56:12.821974 11480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c44cfd31e13747a9bcde8fc9f7bbb365: Bootstrap starting.
I20260924 00:56:12.825743 11480 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c44cfd31e13747a9bcde8fc9f7bbb365: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:12.827139 11480 log.cc:824] T 00000000000000000000000000000000 P c44cfd31e13747a9bcde8fc9f7bbb365: Log is configured to *not* fsync() on all Append() calls
Sep 24 00:56:12 dist-test-slave-mmhs krb5kdc[11400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:56:12.830103 11480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c44cfd31e13747a9bcde8fc9f7bbb365: No bootstrap required, opened a new log
I20260924 00:56:12.837244 11483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:12.815332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55422 (local address 127.8.64.190:43795)
0924 00:56:12.816106 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:12.816119 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:12.816158 (+ 39us) server_negotiation.cc:407] Connection header received
0924 00:56:12.817763 (+ 1605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:12.817785 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:12.818178 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:12.818576 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:12.820399 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:12.821674 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:12.823416 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:12.824144 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:12.824291 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:12.829304 (+ 5013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:12.829313 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:12.829329 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:12.829364 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:12.832750 (+ 3386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:12.833572 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:12.833580 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:12.833586 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:12.833744 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:12.834172 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:12.834176 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:12.834179 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:12.834822 (+ 643us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:12.835182 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:12.835658 (+ 476us) server_negotiation.cc:299] Negotiation successful
0924 00:56:12.836159 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":381,"threads_started":1}
I20260924 00:56:12.838135 11480 raft_consensus.cc:348] T 00000000000000000000000000000000 P c44cfd31e13747a9bcde8fc9f7bbb365 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c44cfd31e13747a9bcde8fc9f7bbb365" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 43795 } }
I20260924 00:56:12.838599 11480 raft_consensus.cc:374] T 00000000000000000000000000000000 P c44cfd31e13747a9bcde8fc9f7bbb365 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:12.838717 11480 raft_consensus.cc:729] T 00000000000000000000000000000000 P c44cfd31e13747a9bcde8fc9f7bbb365 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c44cfd31e13747a9bcde8fc9f7bbb365, State: Initialized, Role: FOLLOWER
I20260924 00:56:12.839299 11480 consensus_queue.cc:260] T 00000000000000000000000000000000 P c44cfd31e13747a9bcde8fc9f7bbb365 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c44cfd31e13747a9bcde8fc9f7bbb365" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 43795 } }
I20260924 00:56:12.839532 11480 raft_consensus.cc:388] T 00000000000000000000000000000000 P c44cfd31e13747a9bcde8fc9f7bbb365 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:56:12.839663 11480 raft_consensus.cc:482] T 00000000000000000000000000000000 P c44cfd31e13747a9bcde8fc9f7bbb365 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:56:12.839862 11480 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c44cfd31e13747a9bcde8fc9f7bbb365 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:12.841482 8450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 00:56:12.842422 11480 raft_consensus.cc:504] T 00000000000000000000000000000000 P c44cfd31e13747a9bcde8fc9f7bbb365 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c44cfd31e13747a9bcde8fc9f7bbb365" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 43795 } }
I20260924 00:56:12.842891 11480 leader_election.cc:302] T 00000000000000000000000000000000 P c44cfd31e13747a9bcde8fc9f7bbb365 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c44cfd31e13747a9bcde8fc9f7bbb365; no voters:
I20260924 00:56:12.843530 11480 leader_election.cc:288] T 00000000000000000000000000000000 P c44cfd31e13747a9bcde8fc9f7bbb365 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:56:12.843640 11485 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c44cfd31e13747a9bcde8fc9f7bbb365 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:56:12.846084 11480 sys_catalog.cc:564] T 00000000000000000000000000000000 P c44cfd31e13747a9bcde8fc9f7bbb365 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:56:12.853473 11485 raft_consensus.cc:686] T 00000000000000000000000000000000 P c44cfd31e13747a9bcde8fc9f7bbb365 [term 1 LEADER]: Becoming Leader. State: Replica: c44cfd31e13747a9bcde8fc9f7bbb365, State: Running, Role: LEADER
I20260924 00:56:12.854161 11485 consensus_queue.cc:237] T 00000000000000000000000000000000 P c44cfd31e13747a9bcde8fc9f7bbb365 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c44cfd31e13747a9bcde8fc9f7bbb365" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 43795 } }
I20260924 00:56:12.857547 11487 sys_catalog.cc:455] T 00000000000000000000000000000000 P c44cfd31e13747a9bcde8fc9f7bbb365 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c44cfd31e13747a9bcde8fc9f7bbb365. Latest consensus state: current_term: 1 leader_uuid: "c44cfd31e13747a9bcde8fc9f7bbb365" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c44cfd31e13747a9bcde8fc9f7bbb365" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 43795 } } }
I20260924 00:56:12.857980 11487 sys_catalog.cc:458] T 00000000000000000000000000000000 P c44cfd31e13747a9bcde8fc9f7bbb365 [sys.catalog]: This master's current role is: LEADER
I20260924 00:56:12.858634 11486 sys_catalog.cc:455] T 00000000000000000000000000000000 P c44cfd31e13747a9bcde8fc9f7bbb365 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c44cfd31e13747a9bcde8fc9f7bbb365" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c44cfd31e13747a9bcde8fc9f7bbb365" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 43795 } } }
I20260924 00:56:12.859095 11486 sys_catalog.cc:458] T 00000000000000000000000000000000 P c44cfd31e13747a9bcde8fc9f7bbb365 [sys.catalog]: This master's current role is: LEADER
W20260924 00:56:12.869844 11500 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c44cfd31e13747a9bcde8fc9f7bbb365: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 00:56:12.869984 11500 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 00:56:12.870107 11501 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:56:12.872293 11501 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:56:12.882596 11501 catalog_manager.cc:1347] Generated new cluster ID: 667ace70292647b48b0c74f83f5a8114
I20260924 00:56:12.882791 11501 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:56:12.890475 11483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:12.883696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55436 (local address 127.8.64.190:43795)
0924 00:56:12.883859 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:12.883867 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:12.883896 (+ 29us) server_negotiation.cc:407] Connection header received
0924 00:56:12.884038 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:12.884050 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:12.884112 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:12.884232 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:12.885230 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:12.885944 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:12.887307 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:12.887742 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:12.887881 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:12.888990 (+ 1109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:12.888995 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:12.889001 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:56:12.889586 (+ 585us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:12.889648 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:56:12.889711 (+ 63us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:12.889829 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:12.889978 (+ 149us) server_negotiation.cc:299] Negotiation successful
0924 00:56:12.890075 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260924 00:56:12.908165 11501 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:56:12.909528 11501 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:56:12.918607 11501 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c44cfd31e13747a9bcde8fc9f7bbb365: Generated new TSK 0
I20260924 00:56:12.919647 11501 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 00:56:12.928820 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 11413
I20260924 00:56:12.930020 11475 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:56:13.019726 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 11413
2026-09-24T00:56:13Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1918 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:56:13 dist-test-slave-mmhs krb5kdc[11514](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:56:13 dist-test-slave-mmhs krb5kdc[11514](info): set up 2 sockets
Sep 24 00:56:13 dist-test-slave-mmhs krb5kdc[11514](info): commencing operation
krb5kdc: starting...
W20260924 00:56:14.116395 8450 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.000s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Sep 24 00:56:14 dist-test-slave-mmhs krb5kdc[11514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:56:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:56:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.64.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
I20260924 00:56:14.313648 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:42115
--webserver_interface=127.8.64.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:36853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:42115
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:56:14.631058 11527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:14.631345 11527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:14.631537 11527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:14.639456 11527 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:56:14.639566 11527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:14.639672 11527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:14.639787 11527 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:56:14.639853 11527 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:56:14.649657 11527 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:36853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:42115
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:42115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:14.651055 11527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:14.652343 11527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:14.664477 11535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:14.664541 11532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:14.664532 11533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:14.665126 11527 server_base.cc:1034] running on GCE node
I20260924 00:56:14.665944 11527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:14.667471 11527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:14.668668 11527 hybrid_clock.cc:648] HybridClock initialized: now 1790211374668633 us; error 35 us; skew 500 ppm
I20260924 00:56:14.669121 11527 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:14 dist-test-slave-mmhs krb5kdc[11514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211374, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:14.672464 11527 init.cc:377] Logged in from keytab as kudu/127.8.64.190@KRBTEST.COM (short username kudu)
I20260924 00:56:14.673897 11527 webserver.cc:460] Webserver started at http://127.8.64.190:40443/ using document root <none> and password file <none>
I20260924 00:56:14.674659 11527 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:14.674790 11527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:14.675113 11527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:56:14.678125 11527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "ac6501c724114fdcbfe1a8f706ce7486"
format_stamp: "Formatted at 2026-09-24 00:56:14 on dist-test-slave-mmhs"
server_key: "cd9272acba038fa170b76e11a0c434c1"
server_key_iv: "4afb1558cbc075ac4b89e0bf9fff9dfb"
server_key_version: "encryptionkey@0"
I20260924 00:56:14.678972 11527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "ac6501c724114fdcbfe1a8f706ce7486"
format_stamp: "Formatted at 2026-09-24 00:56:14 on dist-test-slave-mmhs"
server_key: "cd9272acba038fa170b76e11a0c434c1"
server_key_iv: "4afb1558cbc075ac4b89e0bf9fff9dfb"
server_key_version: "encryptionkey@0"
I20260924 00:56:14.684163 11527 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260924 00:56:14.687716 11542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:14.689057 11527 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260924 00:56:14.689255 11527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "ac6501c724114fdcbfe1a8f706ce7486"
format_stamp: "Formatted at 2026-09-24 00:56:14 on dist-test-slave-mmhs"
server_key: "cd9272acba038fa170b76e11a0c434c1"
server_key_iv: "4afb1558cbc075ac4b89e0bf9fff9dfb"
server_key_version: "encryptionkey@0"
I20260924 00:56:14.689456 11527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:14.716914 11527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:14.720920 11527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:14.721225 11527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:14.739862 11527 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.190:42115
I20260924 00:56:14.739892 11593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.190:42115 every 8 connection(s)
I20260924 00:56:14.741271 11527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 00:56:14.742148 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 11527
I20260924 00:56:14.742404 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 00:56:14.743041 8450 external_mini_cluster.cc:1428] Setting key e7b858869029a58b5a9d443b8aee1eeb
I20260924 00:56:14.745628 11594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 24 00:56:14 dist-test-slave-mmhs krb5kdc[11514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:56:14.759202 11594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac6501c724114fdcbfe1a8f706ce7486: Bootstrap starting.
I20260924 00:56:14.762696 11594 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac6501c724114fdcbfe1a8f706ce7486: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:14.763746 11594 log.cc:824] T 00000000000000000000000000000000 P ac6501c724114fdcbfe1a8f706ce7486: Log is configured to *not* fsync() on all Append() calls
I20260924 00:56:14.765911 11597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:14.744937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48514 (local address 127.8.64.190:42115)
0924 00:56:14.745768 (+ 831us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:14.745784 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:14.745836 (+ 52us) server_negotiation.cc:407] Connection header received
0924 00:56:14.747454 (+ 1618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:14.747475 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:14.747854 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:14.748210 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:14.749968 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:14.751443 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:14.753206 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:14.753902 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:14.754034 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:14.758718 (+ 4684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:14.758726 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:14.758742 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:14.758785 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:14.761882 (+ 3097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:14.762649 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:14.762657 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:14.762663 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:14.762857 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:14.763234 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:14.763241 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:14.763244 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:14.763756 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:14.764089 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:14.764513 (+ 424us) server_negotiation.cc:299] Negotiation successful
0924 00:56:14.764935 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":368,"threads_started":1}
I20260924 00:56:14.766924 11594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac6501c724114fdcbfe1a8f706ce7486: No bootstrap required, opened a new log
I20260924 00:56:14.774791 11594 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac6501c724114fdcbfe1a8f706ce7486 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6501c724114fdcbfe1a8f706ce7486" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42115 } }
I20260924 00:56:14.775190 11594 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac6501c724114fdcbfe1a8f706ce7486 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:14.775308 11594 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac6501c724114fdcbfe1a8f706ce7486 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac6501c724114fdcbfe1a8f706ce7486, State: Initialized, Role: FOLLOWER
I20260924 00:56:14.775915 11594 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac6501c724114fdcbfe1a8f706ce7486 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6501c724114fdcbfe1a8f706ce7486" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42115 } }
I20260924 00:56:14.776171 11594 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac6501c724114fdcbfe1a8f706ce7486 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:56:14.776309 11594 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac6501c724114fdcbfe1a8f706ce7486 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:56:14.776466 11594 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac6501c724114fdcbfe1a8f706ce7486 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:14.778621 11594 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac6501c724114fdcbfe1a8f706ce7486 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6501c724114fdcbfe1a8f706ce7486" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42115 } }
I20260924 00:56:14.779059 11594 leader_election.cc:302] T 00000000000000000000000000000000 P ac6501c724114fdcbfe1a8f706ce7486 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac6501c724114fdcbfe1a8f706ce7486; no voters:
I20260924 00:56:14.779680 11594 leader_election.cc:288] T 00000000000000000000000000000000 P ac6501c724114fdcbfe1a8f706ce7486 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:56:14.779805 11599 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac6501c724114fdcbfe1a8f706ce7486 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:56:14.780953 11599 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac6501c724114fdcbfe1a8f706ce7486 [term 1 LEADER]: Becoming Leader. State: Replica: ac6501c724114fdcbfe1a8f706ce7486, State: Running, Role: LEADER
I20260924 00:56:14.781487 11599 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac6501c724114fdcbfe1a8f706ce7486 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6501c724114fdcbfe1a8f706ce7486" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42115 } }
I20260924 00:56:14.781817 11594 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac6501c724114fdcbfe1a8f706ce7486 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:56:14.784628 11601 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac6501c724114fdcbfe1a8f706ce7486 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac6501c724114fdcbfe1a8f706ce7486. Latest consensus state: current_term: 1 leader_uuid: "ac6501c724114fdcbfe1a8f706ce7486" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6501c724114fdcbfe1a8f706ce7486" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42115 } } }
I20260924 00:56:14.785073 11601 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac6501c724114fdcbfe1a8f706ce7486 [sys.catalog]: This master's current role is: LEADER
I20260924 00:56:14.784679 11600 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac6501c724114fdcbfe1a8f706ce7486 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac6501c724114fdcbfe1a8f706ce7486" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6501c724114fdcbfe1a8f706ce7486" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42115 } } }
I20260924 00:56:14.785540 11600 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac6501c724114fdcbfe1a8f706ce7486 [sys.catalog]: This master's current role is: LEADER
I20260924 00:56:14.791517 11608 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:56:14.796969 11608 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:56:14.806576 11608 catalog_manager.cc:1347] Generated new cluster ID: 4bc01bd60cef4a829658c248eb0a084f
I20260924 00:56:14.806699 11608 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:56:14.820585 11608 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:56:14.822021 11608 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:56:14.836941 11608 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac6501c724114fdcbfe1a8f706ce7486: Generated new TSK 0
I20260924 00:56:14.838016 11608 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.64.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab.
Entry for principal kudu/127.8.64.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab.
I20260924 00:56:14.893875 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.129:0
--local_ip_for_outbound_sockets=127.8.64.129
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:42115
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab
--principal=kudu/127.8.64.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:36853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:56:15.246482 11620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:15.246748 11620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:15.246920 11620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:15.254128 11620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:15.254241 11620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:15.254462 11620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.129
I20260924 00:56:15.263991 11620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:36853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab
--principal=kudu/127.8.64.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:42115
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.129
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:15.265193 11620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:15.266628 11620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:15.279814 11625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:15.279902 11626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:15.279865 11628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:15.281006 11620 server_base.cc:1034] running on GCE node
I20260924 00:56:15.281755 11620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:15.282855 11620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:15.284055 11620 hybrid_clock.cc:648] HybridClock initialized: now 1790211375283992 us; error 65 us; skew 500 ppm
I20260924 00:56:15.284389 11620 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:15 dist-test-slave-mmhs krb5kdc[11514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211375, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:15.287734 11620 init.cc:377] Logged in from keytab as kudu/127.8.64.129@KRBTEST.COM (short username kudu)
I20260924 00:56:15.289081 11620 webserver.cc:460] Webserver started at http://127.8.64.129:40181/ using document root <none> and password file <none>
I20260924 00:56:15.289712 11620 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:15.289812 11620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:15.290040 11620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:56:15.293008 11620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-0/data/instance:
uuid: "48d74a5fdf1a4b89ac260b35c37fe1fe"
format_stamp: "Formatted at 2026-09-24 00:56:15 on dist-test-slave-mmhs"
server_key: "dafd141641a3a9364c56a46f17496dd3"
server_key_iv: "0e4839514a8cb874d812a2d48ff640d3"
server_key_version: "encryptionkey@0"
I20260924 00:56:15.293794 11620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance:
uuid: "48d74a5fdf1a4b89ac260b35c37fe1fe"
format_stamp: "Formatted at 2026-09-24 00:56:15 on dist-test-slave-mmhs"
server_key: "dafd141641a3a9364c56a46f17496dd3"
server_key_iv: "0e4839514a8cb874d812a2d48ff640d3"
server_key_version: "encryptionkey@0"
I20260924 00:56:15.298736 11620 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260924 00:56:15.301925 11635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:15.303288 11620 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 00:56:15.303462 11620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-0/wal
uuid: "48d74a5fdf1a4b89ac260b35c37fe1fe"
format_stamp: "Formatted at 2026-09-24 00:56:15 on dist-test-slave-mmhs"
server_key: "dafd141641a3a9364c56a46f17496dd3"
server_key_iv: "0e4839514a8cb874d812a2d48ff640d3"
server_key_version: "encryptionkey@0"
I20260924 00:56:15.303638 11620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:15.320360 11620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:15.324520 11620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:15.324822 11620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:15.325744 11620 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:56:15.327134 11620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:56:15.327214 11620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:15.327353 11620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:56:15.327414 11620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:15.359169 11620 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.129:36869
I20260924 00:56:15.359253 11748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.129:36869 every 8 connection(s)
I20260924 00:56:15.360427 11620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
I20260924 00:56:15.363605 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 11620
I20260924 00:56:15.363834 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance
I20260924 00:56:15.364710 8450 external_mini_cluster.cc:1428] Setting key f0d73e3c6b89831c667c8e453d6347f9
Sep 24 00:56:15 dist-test-slave-mmhs krb5kdc[11514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211375, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:56:15.381629 11751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:15.362898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:58675 (local address 127.8.64.190:42115)
0924 00:56:15.363360 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:15.363365 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:15.364449 (+ 1084us) server_negotiation.cc:407] Connection header received
0924 00:56:15.366453 (+ 2004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:15.366458 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:15.366546 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:15.366659 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:15.368092 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:15.368777 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:15.370270 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:15.370700 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:15.370830 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:15.374453 (+ 3623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:15.374461 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:15.374467 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:15.374515 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:15.378203 (+ 3688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:15.378998 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:15.379007 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:15.379013 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:15.379158 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:15.379560 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:15.379568 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:15.379575 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:15.379948 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:15.380110 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:15.380865 (+ 755us) server_negotiation.cc:299] Negotiation successful
0924 00:56:15.381130 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":259,"threads_started":1}
I20260924 00:56:15.382380 11752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:15.363466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42115 (local address 127.8.64.129:58675)
0924 00:56:15.364296 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:15.364332 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:15.366123 (+ 1791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:15.366888 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:15.366896 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:15.367371 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:15.367926 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:15.367943 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:15.368930 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:15.368935 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:15.370118 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:15.370129 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:15.370914 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:15.370920 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:15.371149 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:15.371926 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:15.371949 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:15.374182 (+ 2233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:15.378410 (+ 4228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:15.378416 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:15.378432 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:15.378863 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:15.379271 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:15.379274 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:15.379277 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:15.379452 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:15.380079 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:15.380085 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:15.380443 (+ 358us) client_negotiation.cc:769] Sending connection context
0924 00:56:15.380731 (+ 288us) client_negotiation.cc:240] Negotiation successful
0924 00:56:15.381104 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":365,"threads_started":1}
I20260924 00:56:15.384052 11749 heartbeater.cc:347] Connected to a master server at 127.8.64.190:42115
I20260924 00:56:15.384357 11749 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.8.64.130@KRBTEST.COM; defaulting to no policy
I20260924 00:56:15.385120 11749 heartbeater.cc:511] Master 127.8.64.190:42115 requested a full tablet report, sending...
I20260924 00:56:15.387202 11559 ts_manager.cc:194] Registered new tserver with Master: 48d74a5fdf1a4b89ac260b35c37fe1fe (127.8.64.129:36869)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.130@KRBTEST.COM" created.
I20260924 00:56:15.388648 11559 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.129@KRBTEST.COM'} at 127.8.64.129:58675
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab.
Entry for principal kudu/127.8.64.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab.
I20260924 00:56:15.413096 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.130:0
--local_ip_for_outbound_sockets=127.8.64.130
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:42115
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab
--principal=kudu/127.8.64.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:36853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:56:15.732590 11755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:15.732857 11755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:15.733029 11755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:15.740203 11755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:15.740300 11755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:15.740479 11755 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.130
I20260924 00:56:15.750020 11755 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:36853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab
--principal=kudu/127.8.64.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:42115
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.130
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:15.751260 11755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:15.752524 11755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:15.765386 11760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:15.765331 11761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:15.765350 11763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:15.766321 11755 server_base.cc:1034] running on GCE node
I20260924 00:56:15.766924 11755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:15.767815 11755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:15.769052 11755 hybrid_clock.cc:648] HybridClock initialized: now 1790211375769011 us; error 48 us; skew 500 ppm
I20260924 00:56:15.769376 11755 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:15 dist-test-slave-mmhs krb5kdc[11514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211375, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:15.772562 11755 init.cc:377] Logged in from keytab as kudu/127.8.64.130@KRBTEST.COM (short username kudu)
I20260924 00:56:15.773870 11755 webserver.cc:460] Webserver started at http://127.8.64.130:39277/ using document root <none> and password file <none>
I20260924 00:56:15.774545 11755 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:15.774647 11755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:15.774899 11755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:56:15.777667 11755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-1/data/instance:
uuid: "35b2d348a3f9482099ae97dd54e0a20a"
format_stamp: "Formatted at 2026-09-24 00:56:15 on dist-test-slave-mmhs"
server_key: "12241da9746861e0ca730cd0e6213c79"
server_key_iv: "34c5f48b51550586a47e9ef3986907ed"
server_key_version: "encryptionkey@0"
I20260924 00:56:15.778452 11755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance:
uuid: "35b2d348a3f9482099ae97dd54e0a20a"
format_stamp: "Formatted at 2026-09-24 00:56:15 on dist-test-slave-mmhs"
server_key: "12241da9746861e0ca730cd0e6213c79"
server_key_iv: "34c5f48b51550586a47e9ef3986907ed"
server_key_version: "encryptionkey@0"
I20260924 00:56:15.783564 11755 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260924 00:56:15.787214 11770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:15.788481 11755 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260924 00:56:15.788609 11755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-1/wal
uuid: "35b2d348a3f9482099ae97dd54e0a20a"
format_stamp: "Formatted at 2026-09-24 00:56:15 on dist-test-slave-mmhs"
server_key: "12241da9746861e0ca730cd0e6213c79"
server_key_iv: "34c5f48b51550586a47e9ef3986907ed"
server_key_version: "encryptionkey@0"
I20260924 00:56:15.788764 11755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:15.827610 11755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:15.831977 11755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:15.832242 11755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:15.833156 11755 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:56:15.834457 11755 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:56:15.834528 11755 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:15.834612 11755 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:56:15.834656 11755 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:15.867486 11755 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.130:45481
I20260924 00:56:15.867534 11883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.130:45481 every 8 connection(s)
I20260924 00:56:15.868736 11755 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
I20260924 00:56:15.871507 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 11755
I20260924 00:56:15.871691 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance
I20260924 00:56:15.872344 8450 external_mini_cluster.cc:1428] Setting key 380e37835e424bcae05926facc0b1653
Sep 24 00:56:15 dist-test-slave-mmhs krb5kdc[11514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211375, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.130@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:56:15.890007 11751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:15.871107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:48499 (local address 127.8.64.190:42115)
0924 00:56:15.871816 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:15.871823 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:15.872619 (+ 796us) server_negotiation.cc:407] Connection header received
0924 00:56:15.874704 (+ 2085us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:15.874709 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:15.874800 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:15.874904 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:15.876357 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:15.877051 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:15.878640 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:15.879137 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:15.879275 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:15.882858 (+ 3583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:15.882867 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:15.882876 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:15.882921 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:15.886542 (+ 3621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:15.887308 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:15.887316 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:15.887323 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:15.887480 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:15.888085 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:15.888091 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:15.888095 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:15.888490 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:15.888663 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:15.889359 (+ 696us) server_negotiation.cc:299] Negotiation successful
0924 00:56:15.889620 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558}
I20260924 00:56:15.890872 11886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:15.871666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42115 (local address 127.8.64.130:48499)
0924 00:56:15.872427 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:15.872483 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:15.874447 (+ 1964us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:15.875129 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:15.875137 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:15.875640 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:15.876193 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:15.876209 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:15.877210 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:15.877216 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:15.878496 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:15.878508 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:15.879402 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:15.879411 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:15.879741 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:15.880448 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:15.880471 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:15.882638 (+ 2167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:15.886701 (+ 4063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:15.886707 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:15.886722 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:15.887161 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:15.887620 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:15.887658 (+ 38us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:15.887664 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:15.887943 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:15.888658 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:15.888664 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:15.888974 (+ 310us) client_negotiation.cc:769] Sending connection context
0924 00:56:15.889206 (+ 232us) client_negotiation.cc:240] Negotiation successful
0924 00:56:15.889618 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":317,"threads_started":1}
WARNING: no policy specified for kudu/127.8.64.131@KRBTEST.COM; defaulting to no policy
I20260924 00:56:15.892526 11884 heartbeater.cc:347] Connected to a master server at 127.8.64.190:42115
I20260924 00:56:15.892900 11884 heartbeater.cc:464] Registering TS with master...
I20260924 00:56:15.893750 11884 heartbeater.cc:511] Master 127.8.64.190:42115 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.131@KRBTEST.COM" created.
I20260924 00:56:15.895329 11559 ts_manager.cc:194] Registered new tserver with Master: 35b2d348a3f9482099ae97dd54e0a20a (127.8.64.130:45481)
I20260924 00:56:15.896241 11559 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.130@KRBTEST.COM'} at 127.8.64.130:48499
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab.
Entry for principal kudu/127.8.64.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab.
I20260924 00:56:15.919920 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.131:0
--local_ip_for_outbound_sockets=127.8.64.131
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:42115
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab
--principal=kudu/127.8.64.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:36853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:56:16.241381 11889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:16.241655 11889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:16.241748 11889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:16.248852 11889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:16.248961 11889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:16.249073 11889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.131
I20260924 00:56:16.258670 11889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:36853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab
--principal=kudu/127.8.64.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:42115
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.131
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:16.259852 11889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:16.261063 11889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:16.273666 11894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:16.273840 11895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:16.274260 11897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:16.275029 11889 server_base.cc:1034] running on GCE node
I20260924 00:56:16.275792 11889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:16.276660 11889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:16.277905 11889 hybrid_clock.cc:648] HybridClock initialized: now 1790211376277844 us; error 52 us; skew 500 ppm
I20260924 00:56:16.278264 11889 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:16 dist-test-slave-mmhs krb5kdc[11514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211376, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:16.281677 11889 init.cc:377] Logged in from keytab as kudu/127.8.64.131@KRBTEST.COM (short username kudu)
I20260924 00:56:16.283304 11889 webserver.cc:460] Webserver started at http://127.8.64.131:36935/ using document root <none> and password file <none>
I20260924 00:56:16.283957 11889 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:16.284086 11889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:16.284395 11889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:56:16.287441 11889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-2/data/instance:
uuid: "eec96169542d4e14a17f85a5140d3b06"
format_stamp: "Formatted at 2026-09-24 00:56:16 on dist-test-slave-mmhs"
server_key: "dcc60e4edb18c3fedbfe0b27ccfd76d2"
server_key_iv: "f0e51c640305f2e2133a70ba1d4412dd"
server_key_version: "encryptionkey@0"
I20260924 00:56:16.288203 11889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance:
uuid: "eec96169542d4e14a17f85a5140d3b06"
format_stamp: "Formatted at 2026-09-24 00:56:16 on dist-test-slave-mmhs"
server_key: "dcc60e4edb18c3fedbfe0b27ccfd76d2"
server_key_iv: "f0e51c640305f2e2133a70ba1d4412dd"
server_key_version: "encryptionkey@0"
I20260924 00:56:16.293476 11889 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260924 00:56:16.297173 11904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:16.298739 11889 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 00:56:16.298923 11889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-2/wal
uuid: "eec96169542d4e14a17f85a5140d3b06"
format_stamp: "Formatted at 2026-09-24 00:56:16 on dist-test-slave-mmhs"
server_key: "dcc60e4edb18c3fedbfe0b27ccfd76d2"
server_key_iv: "f0e51c640305f2e2133a70ba1d4412dd"
server_key_version: "encryptionkey@0"
I20260924 00:56:16.299124 11889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:16.326331 11889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:16.330449 11889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:16.330757 11889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:16.331768 11889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:56:16.333186 11889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:56:16.333258 11889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:16.333411 11889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:56:16.333462 11889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:16.367208 11889 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.131:46843
I20260924 00:56:16.367305 12017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.131:46843 every 8 connection(s)
I20260924 00:56:16.368418 11889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
I20260924 00:56:16.378844 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 11889
I20260924 00:56:16.379081 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance
I20260924 00:56:16.379688 8450 external_mini_cluster.cc:1428] Setting key f6ec2464f132e9d4f1d4210de6d75cf8
Sep 24 00:56:16 dist-test-slave-mmhs krb5kdc[11514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211376, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.131@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:56:16.388375 11751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:16.370909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:57779 (local address 127.8.64.190:42115)
0924 00:56:16.371104 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:16.371110 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:16.372200 (+ 1090us) server_negotiation.cc:407] Connection header received
0924 00:56:16.373913 (+ 1713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:16.373917 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:16.373983 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:16.374078 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:16.375577 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:16.376319 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:16.377861 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:16.378361 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:16.378552 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:16.382728 (+ 4176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:16.382734 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:16.382739 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:16.382786 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:16.385474 (+ 2688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:16.386244 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:16.386247 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:16.386250 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:16.386425 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:16.386849 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:16.386852 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:16.386854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:16.387093 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:16.387206 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:16.387977 (+ 771us) server_negotiation.cc:299] Negotiation successful
0924 00:56:16.388135 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260924 00:56:16.389376 12020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:16.371388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42115 (local address 127.8.64.131:57779)
0924 00:56:16.372048 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:16.372090 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:16.373669 (+ 1579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:16.374268 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:16.374276 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:16.374856 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:16.375391 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:16.375407 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:16.376464 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:16.376468 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:16.377703 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:16.377714 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:16.378612 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:16.378617 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:16.378817 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:16.379534 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:16.379555 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:16.382390 (+ 2835us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:16.385614 (+ 3224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:16.385621 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:16.385633 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:16.386142 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:16.386531 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:16.386535 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:16.386544 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:16.386751 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:16.387247 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:16.387254 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:16.387604 (+ 350us) client_negotiation.cc:769] Sending connection context
0924 00:56:16.387862 (+ 258us) client_negotiation.cc:240] Negotiation successful
0924 00:56:16.388264 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":259,"threads_started":1}
I20260924 00:56:16.390856 12018 heartbeater.cc:347] Connected to a master server at 127.8.64.190:42115
I20260924 00:56:16.391173 12018 heartbeater.cc:464] Registering TS with master...
I20260924 00:56:16.391515 11749 heartbeater.cc:503] Master 127.8.64.190:42115 was elected leader, sending a full tablet report...
I20260924 00:56:16.392050 12018 heartbeater.cc:511] Master 127.8.64.190:42115 requested a full tablet report, sending...
I20260924 00:56:16.393678 11559 ts_manager.cc:194] Registered new tserver with Master: eec96169542d4e14a17f85a5140d3b06 (127.8.64.131:46843)
I20260924 00:56:16.394761 11559 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.131@KRBTEST.COM'} at 127.8.64.131:57779
I20260924 00:56:16.395475 8450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 24 00:56:16 dist-test-slave-mmhs krb5kdc[11514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211376, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 24 00:56:16 dist-test-slave-mmhs krb5kdc[11514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211376, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:56:16.443995 11751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:16.430609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48528 (local address 127.8.64.190:42115)
0924 00:56:16.430773 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:16.430778 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:16.431398 (+ 620us) server_negotiation.cc:407] Connection header received
0924 00:56:16.431586 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:16.431591 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:16.431680 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:16.431773 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:16.433080 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:16.433799 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:16.435105 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:16.435487 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:16.435580 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:16.439313 (+ 3733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:16.439318 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:16.439321 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:16.439350 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:16.441662 (+ 2312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:16.442281 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:16.442285 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:16.442287 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:16.442464 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:16.442860 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:16.442864 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:16.442866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:16.443155 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:16.443327 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:16.443624 (+ 297us) server_negotiation.cc:299] Negotiation successful
0924 00:56:16.443776 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260924 00:56:16.444509 11558 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:48528
I20260924 00:56:16.446218 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 11620
Sep 24 00:56:16 dist-test-slave-mmhs krb5kdc[11514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211376, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.8.64.129@KRBTEST.COM
I20260924 00:56:16.461113 12029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:16.447265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51554 (local address 127.8.64.129:36869)
0924 00:56:16.447849 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:16.447856 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:16.447879 (+ 23us) server_negotiation.cc:407] Connection header received
0924 00:56:16.447929 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:16.447935 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:16.448141 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:16.448258 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:16.449337 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:16.450363 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:16.451452 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:16.452006 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:16.452109 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:16.455915 (+ 3806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:16.455924 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:16.455937 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:16.455971 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:16.458768 (+ 2797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:16.459304 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:16.459310 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:16.459313 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:16.459418 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:16.459771 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:16.459775 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:16.459777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:16.460137 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:16.460360 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:16.460623 (+ 263us) server_negotiation.cc:299] Negotiation successful
0924 00:56:16.460808 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":299,"threads_started":1}
W20260924 00:56:16.462612 11743 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:51554
W20260924 00:56:16.463656 8450 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 11620: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260924 00:56:16.463822 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 11620
I20260924 00:56:16.479537 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 11755
Sep 24 00:56:16 dist-test-slave-mmhs krb5kdc[11514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211376, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.8.64.130@KRBTEST.COM
I20260924 00:56:16.495733 12030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:16.480384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33476 (local address 127.8.64.130:45481)
0924 00:56:16.480953 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:16.480960 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:16.480985 (+ 25us) server_negotiation.cc:407] Connection header received
0924 00:56:16.481041 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:16.481047 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:16.481255 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:16.481423 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:16.482727 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:16.483720 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:16.485026 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:16.485580 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:16.485789 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:16.489724 (+ 3935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:16.489734 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:16.489749 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:16.489789 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:16.492594 (+ 2805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:16.493461 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:16.493472 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:16.493478 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:16.493669 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:16.494121 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:16.494125 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:16.494150 (+ 25us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:16.494601 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:16.494835 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:16.495189 (+ 354us) server_negotiation.cc:299] Negotiation successful
0924 00:56:16.495402 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":320,"threads_started":1}
W20260924 00:56:16.496999 11879 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:33476
W20260924 00:56:16.498159 8450 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 11755: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260924 00:56:16.498339 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 11755
I20260924 00:56:16.514173 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 11889
Sep 24 00:56:16 dist-test-slave-mmhs krb5kdc[11514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211376, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.8.64.131@KRBTEST.COM
I20260924 00:56:16.531042 12031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:16.515177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46666 (local address 127.8.64.131:46843)
0924 00:56:16.515795 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:16.515801 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:16.515829 (+ 28us) server_negotiation.cc:407] Connection header received
0924 00:56:16.515885 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:16.515891 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:16.516110 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:16.516283 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:16.517587 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:16.518669 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:16.520058 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:16.520609 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:16.520752 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:16.524909 (+ 4157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:16.524920 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:16.524934 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:16.524965 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:16.527881 (+ 2916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:16.528650 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:16.528665 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:16.528667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:16.528798 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:16.529192 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:16.529199 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:16.529203 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:16.529562 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:16.529893 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:16.530212 (+ 319us) server_negotiation.cc:299] Negotiation successful
0924 00:56:16.530640 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":315,"threads_started":1}
W20260924 00:56:16.532312 12013 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:46666
W20260924 00:56:16.533366 8450 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 11889: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260924 00:56:16.533522 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 11889
I20260924 00:56:16.549094 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 11527
I20260924 00:56:16.550438 11589 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:56:16.654333 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 11527
2026-09-24T00:56:16Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3640 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:56:16 dist-test-slave-mmhs krb5kdc[12035](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:56:16 dist-test-slave-mmhs krb5kdc[12035](info): set up 2 sockets
Sep 24 00:56:16 dist-test-slave-mmhs krb5kdc[12035](info): commencing operation
krb5kdc: starting...
W20260924 00:56:17.757222 8450 mini_kdc.cc:121] Time spent starting KDC: real 1.062s user 0.002s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Sep 24 00:56:17 dist-test-slave-mmhs krb5kdc[12035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211377, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:56:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:56:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.64.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
I20260924 00:56:17.963344 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:46791
--webserver_interface=127.8.64.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:40885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:46791
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:56:18.280562 12048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:18.280835 12048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:18.280990 12048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:18.288071 12048 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:56:18.288169 12048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:18.288251 12048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:18.288357 12048 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:56:18.288435 12048 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:56:18.298120 12048 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:40885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:46791
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:46791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:18.299372 12048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:18.300676 12048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:18.312126 12053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:18.312294 12056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:18.312368 12054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:18.313772 12048 server_base.cc:1034] running on GCE node
I20260924 00:56:18.314533 12048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:18.315850 12048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:18.317042 12048 hybrid_clock.cc:648] HybridClock initialized: now 1790211378316996 us; error 33 us; skew 500 ppm
I20260924 00:56:18.317385 12048 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:18 dist-test-slave-mmhs krb5kdc[12035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211378, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:18.326242 12048 init.cc:377] Logged in from keytab as kudu/127.8.64.190@KRBTEST.COM (short username kudu)
I20260924 00:56:18.328290 12048 webserver.cc:460] Webserver started at http://127.8.64.190:39843/ using document root <none> and password file <none>
I20260924 00:56:18.328984 12048 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:18.329084 12048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:18.329355 12048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:56:18.332352 12048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "e5ef35f4d3644681b776cbe8b47575d2"
format_stamp: "Formatted at 2026-09-24 00:56:18 on dist-test-slave-mmhs"
server_key: "e3455271c22703dd7f12ea5810a005d6"
server_key_iv: "32f62a4ef3ca6cf8daa8488569d47d8b"
server_key_version: "encryptionkey@0"
I20260924 00:56:18.333079 12048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "e5ef35f4d3644681b776cbe8b47575d2"
format_stamp: "Formatted at 2026-09-24 00:56:18 on dist-test-slave-mmhs"
server_key: "e3455271c22703dd7f12ea5810a005d6"
server_key_iv: "32f62a4ef3ca6cf8daa8488569d47d8b"
server_key_version: "encryptionkey@0"
I20260924 00:56:18.338071 12048 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260924 00:56:18.341398 12063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:18.342880 12048 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 00:56:18.343053 12048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "e5ef35f4d3644681b776cbe8b47575d2"
format_stamp: "Formatted at 2026-09-24 00:56:18 on dist-test-slave-mmhs"
server_key: "e3455271c22703dd7f12ea5810a005d6"
server_key_iv: "32f62a4ef3ca6cf8daa8488569d47d8b"
server_key_version: "encryptionkey@0"
I20260924 00:56:18.343274 12048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:18.362794 12048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:18.366677 12048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:18.366952 12048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:18.385543 12048 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.190:46791
I20260924 00:56:18.385578 12115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.190:46791 every 8 connection(s)
I20260924 00:56:18.386909 12048 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 00:56:18.390519 12116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:18.391932 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 12048
I20260924 00:56:18.392154 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 00:56:18.393010 8450 external_mini_cluster.cc:1428] Setting key c96f785be80d29f75538c0723a8a2ffc
I20260924 00:56:18.404984 12116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5ef35f4d3644681b776cbe8b47575d2: Bootstrap starting.
Sep 24 00:56:18 dist-test-slave-mmhs krb5kdc[12035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211377, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:56:18.408630 12116 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5ef35f4d3644681b776cbe8b47575d2: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:18.409727 12116 log.cc:824] T 00000000000000000000000000000000 P e5ef35f4d3644681b776cbe8b47575d2: Log is configured to *not* fsync() on all Append() calls
I20260924 00:56:18.412845 12116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5ef35f4d3644681b776cbe8b47575d2: No bootstrap required, opened a new log
I20260924 00:56:18.417551 12119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:18.395120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51936 (local address 127.8.64.190:46791)
0924 00:56:18.395976 (+ 856us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:18.395990 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:18.396033 (+ 43us) server_negotiation.cc:407] Connection header received
0924 00:56:18.397604 (+ 1571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:18.397626 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:18.398037 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:18.398429 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:18.400150 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:18.401451 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:18.403248 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:18.403884 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:18.404009 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:18.408465 (+ 4456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:18.408473 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:18.408488 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:18.408525 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:18.412115 (+ 3590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:18.412766 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:18.412773 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:18.412781 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:18.413004 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:18.413367 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:18.413377 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:18.413381 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:18.414067 (+ 686us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:18.414398 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:18.415742 (+ 1344us) server_negotiation.cc:299] Negotiation successful
0924 00:56:18.416223 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":400,"threads_started":1}
I20260924 00:56:18.422756 12116 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5ef35f4d3644681b776cbe8b47575d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5ef35f4d3644681b776cbe8b47575d2" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 46791 } }
I20260924 00:56:18.423116 12116 raft_consensus.cc:374] T 00000000000000000000000000000000 P e5ef35f4d3644681b776cbe8b47575d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:18.423234 12116 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5ef35f4d3644681b776cbe8b47575d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5ef35f4d3644681b776cbe8b47575d2, State: Initialized, Role: FOLLOWER
I20260924 00:56:18.423852 12116 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5ef35f4d3644681b776cbe8b47575d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5ef35f4d3644681b776cbe8b47575d2" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 46791 } }
I20260924 00:56:18.424091 12116 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5ef35f4d3644681b776cbe8b47575d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:56:18.424238 12116 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5ef35f4d3644681b776cbe8b47575d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:56:18.424412 12116 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5ef35f4d3644681b776cbe8b47575d2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:18.426569 12116 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5ef35f4d3644681b776cbe8b47575d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5ef35f4d3644681b776cbe8b47575d2" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 46791 } }
I20260924 00:56:18.427030 12116 leader_election.cc:302] T 00000000000000000000000000000000 P e5ef35f4d3644681b776cbe8b47575d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5ef35f4d3644681b776cbe8b47575d2; no voters:
I20260924 00:56:18.427595 12116 leader_election.cc:288] T 00000000000000000000000000000000 P e5ef35f4d3644681b776cbe8b47575d2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:56:18.427774 12121 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5ef35f4d3644681b776cbe8b47575d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:56:18.428417 12121 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5ef35f4d3644681b776cbe8b47575d2 [term 1 LEADER]: Becoming Leader. State: Replica: e5ef35f4d3644681b776cbe8b47575d2, State: Running, Role: LEADER
I20260924 00:56:18.429075 12121 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5ef35f4d3644681b776cbe8b47575d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5ef35f4d3644681b776cbe8b47575d2" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 46791 } }
I20260924 00:56:18.429814 12116 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5ef35f4d3644681b776cbe8b47575d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:56:18.432109 12122 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5ef35f4d3644681b776cbe8b47575d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e5ef35f4d3644681b776cbe8b47575d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5ef35f4d3644681b776cbe8b47575d2" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 46791 } } }
I20260924 00:56:18.432541 12122 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5ef35f4d3644681b776cbe8b47575d2 [sys.catalog]: This master's current role is: LEADER
I20260924 00:56:18.434530 12123 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5ef35f4d3644681b776cbe8b47575d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5ef35f4d3644681b776cbe8b47575d2. Latest consensus state: current_term: 1 leader_uuid: "e5ef35f4d3644681b776cbe8b47575d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5ef35f4d3644681b776cbe8b47575d2" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 46791 } } }
I20260924 00:56:18.434927 12123 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5ef35f4d3644681b776cbe8b47575d2 [sys.catalog]: This master's current role is: LEADER
I20260924 00:56:18.436401 12130 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:56:18.440529 12130 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:56:18.449426 12130 catalog_manager.cc:1347] Generated new cluster ID: a2e22962c6bc4864b7e701c2c9525183
I20260924 00:56:18.449538 12130 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:56:18.478721 12130 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:56:18.480060 12130 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:56:18.489296 12130 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e5ef35f4d3644681b776cbe8b47575d2: Generated new TSK 0
I20260924 00:56:18.490175 12130 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.64.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab.
Entry for principal kudu/127.8.64.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab.
I20260924 00:56:18.546661 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.129:0
--local_ip_for_outbound_sockets=127.8.64.129
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:46791
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab
--principal=kudu/127.8.64.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:40885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:56:18.869401 12142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:18.869715 12142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:18.869881 12142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:18.876986 12142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:18.877127 12142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:18.877292 12142 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.129
I20260924 00:56:18.886698 12142 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:40885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab
--principal=kudu/127.8.64.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:46791
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.129
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:18.887918 12142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:18.889189 12142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:18.901858 12148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:18.902073 12147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:18.902455 12150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:18.902689 12142 server_base.cc:1034] running on GCE node
I20260924 00:56:18.903276 12142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:18.904153 12142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:18.905366 12142 hybrid_clock.cc:648] HybridClock initialized: now 1790211378905325 us; error 35 us; skew 500 ppm
I20260924 00:56:18.905694 12142 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:18 dist-test-slave-mmhs krb5kdc[12035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211378, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:18.908655 12142 init.cc:377] Logged in from keytab as kudu/127.8.64.129@KRBTEST.COM (short username kudu)
I20260924 00:56:18.910645 12142 webserver.cc:460] Webserver started at http://127.8.64.129:44507/ using document root <none> and password file <none>
I20260924 00:56:18.911441 12142 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:18.911578 12142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:18.911885 12142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:56:18.914788 12142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-0/data/instance:
uuid: "fd66d7bda6844305ae0515f77aa388c5"
format_stamp: "Formatted at 2026-09-24 00:56:18 on dist-test-slave-mmhs"
server_key: "22560de5c91b91ea7e5b5b1da0911ea7"
server_key_iv: "9ec656fa5b4c54af9219db64f78a6123"
server_key_version: "encryptionkey@0"
I20260924 00:56:18.915555 12142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance:
uuid: "fd66d7bda6844305ae0515f77aa388c5"
format_stamp: "Formatted at 2026-09-24 00:56:18 on dist-test-slave-mmhs"
server_key: "22560de5c91b91ea7e5b5b1da0911ea7"
server_key_iv: "9ec656fa5b4c54af9219db64f78a6123"
server_key_version: "encryptionkey@0"
I20260924 00:56:18.920614 12142 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260924 00:56:18.923931 12157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:18.925289 12142 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260924 00:56:18.925458 12142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-0/wal
uuid: "fd66d7bda6844305ae0515f77aa388c5"
format_stamp: "Formatted at 2026-09-24 00:56:18 on dist-test-slave-mmhs"
server_key: "22560de5c91b91ea7e5b5b1da0911ea7"
server_key_iv: "9ec656fa5b4c54af9219db64f78a6123"
server_key_version: "encryptionkey@0"
I20260924 00:56:18.925659 12142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:18.939088 12142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:18.942806 12142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:18.943082 12142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:18.943950 12142 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:56:18.945176 12142 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:56:18.945276 12142 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:18.945423 12142 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:56:18.945498 12142 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:18.977916 12142 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.129:45987
I20260924 00:56:18.977941 12269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.129:45987 every 8 connection(s)
I20260924 00:56:18.979214 12142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
I20260924 00:56:18.988404 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 12142
I20260924 00:56:18.988636 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance
I20260924 00:56:18.989409 8450 external_mini_cluster.cc:1428] Setting key 087c27cfe331bbc0547171378abb348d
Sep 24 00:56:18 dist-test-slave-mmhs krb5kdc[12035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211378, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:56:19.000289 12272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:18.981844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:56591 (local address 127.8.64.190:46791)
0924 00:56:18.982250 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:18.982255 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:18.983304 (+ 1049us) server_negotiation.cc:407] Connection header received
0924 00:56:18.985310 (+ 2006us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:18.985315 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:18.985384 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:18.985492 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:18.987041 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:18.987837 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:18.990214 (+ 2377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:18.990838 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:18.990962 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:18.994327 (+ 3365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:18.994334 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:18.994340 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:18.994382 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:18.997153 (+ 2771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:18.997921 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:18.997925 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:18.997928 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:18.998049 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:18.998616 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:18.998620 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:18.998622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:18.998909 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:18.999025 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:18.999881 (+ 856us) server_negotiation.cc:299] Negotiation successful
0924 00:56:19.000038 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":208,"threads_started":1}
I20260924 00:56:19.001812 12273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:18.982405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:46791 (local address 127.8.64.129:56591)
0924 00:56:18.983143 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:18.983182 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:18.985037 (+ 1855us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:18.985677 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:18.985685 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:18.986197 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:18.986854 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:18.986872 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:18.988002 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:18.988046 (+ 44us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:18.990077 (+ 2031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:18.990088 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:18.991068 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:18.991073 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:18.991261 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:18.992032 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:18.992054 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:18.994079 (+ 2025us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:18.997310 (+ 3231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:18.997317 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:18.997331 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:18.997748 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:18.998159 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:18.998167 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:18.998173 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:18.998440 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:18.999046 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:18.999057 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:18.999504 (+ 447us) client_negotiation.cc:769] Sending connection context
0924 00:56:18.999799 (+ 295us) client_negotiation.cc:240] Negotiation successful
0924 00:56:19.000390 (+ 591us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":295,"threads_started":1}
I20260924 00:56:19.003422 12270 heartbeater.cc:347] Connected to a master server at 127.8.64.190:46791
I20260924 00:56:19.003731 12270 heartbeater.cc:464] Registering TS with master...
I20260924 00:56:19.004433 12270 heartbeater.cc:511] Master 127.8.64.190:46791 requested a full tablet report, sending...
I20260924 00:56:19.006569 12080 ts_manager.cc:194] Registered new tserver with Master: fd66d7bda6844305ae0515f77aa388c5 (127.8.64.129:45987)
I20260924 00:56:19.007934 12080 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.129@KRBTEST.COM'} at 127.8.64.129:56591
WARNING: no policy specified for kudu/127.8.64.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab.
Entry for principal kudu/127.8.64.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab.
I20260924 00:56:19.038362 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.130:0
--local_ip_for_outbound_sockets=127.8.64.130
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:46791
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab
--principal=kudu/127.8.64.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:40885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:56:19.357318 12276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:19.357596 12276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:19.357751 12276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:19.365027 12276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:19.365134 12276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:19.365330 12276 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.130
I20260924 00:56:19.374692 12276 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:40885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab
--principal=kudu/127.8.64.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:46791
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.130
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:19.375963 12276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:19.377195 12276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:19.392130 12281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:19.392159 12282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:19.393051 12284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:19.393550 12276 server_base.cc:1034] running on GCE node
I20260924 00:56:19.394182 12276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:19.395262 12276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:19.396492 12276 hybrid_clock.cc:648] HybridClock initialized: now 1790211379396427 us; error 57 us; skew 500 ppm
I20260924 00:56:19.396816 12276 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:19 dist-test-slave-mmhs krb5kdc[12035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211379, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:19.399966 12276 init.cc:377] Logged in from keytab as kudu/127.8.64.130@KRBTEST.COM (short username kudu)
I20260924 00:56:19.403358 12276 webserver.cc:460] Webserver started at http://127.8.64.130:45877/ using document root <none> and password file <none>
I20260924 00:56:19.404036 12276 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:19.404171 12276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:19.404464 12276 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:56:19.407336 12276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-1/data/instance:
uuid: "f2a21b5a36cf4551b113db9618caa235"
format_stamp: "Formatted at 2026-09-24 00:56:19 on dist-test-slave-mmhs"
server_key: "0fc97ac6cadf37811c6c66cec4258626"
server_key_iv: "68c03c57a5913874a09f7e617d4d9282"
server_key_version: "encryptionkey@0"
I20260924 00:56:19.408097 12276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance:
uuid: "f2a21b5a36cf4551b113db9618caa235"
format_stamp: "Formatted at 2026-09-24 00:56:19 on dist-test-slave-mmhs"
server_key: "0fc97ac6cadf37811c6c66cec4258626"
server_key_iv: "68c03c57a5913874a09f7e617d4d9282"
server_key_version: "encryptionkey@0"
I20260924 00:56:19.413043 12276 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260924 00:56:19.416491 12291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:19.417826 12276 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260924 00:56:19.418013 12276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-1/wal
uuid: "f2a21b5a36cf4551b113db9618caa235"
format_stamp: "Formatted at 2026-09-24 00:56:19 on dist-test-slave-mmhs"
server_key: "0fc97ac6cadf37811c6c66cec4258626"
server_key_iv: "68c03c57a5913874a09f7e617d4d9282"
server_key_version: "encryptionkey@0"
I20260924 00:56:19.418222 12276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:19.438796 12276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:19.442580 12276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:19.442838 12276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:19.443717 12276 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:56:19.444926 12276 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:56:19.444998 12276 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:19.445087 12276 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:56:19.445132 12276 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:19.477708 12276 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.130:37139
I20260924 00:56:19.477805 12403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.130:37139 every 8 connection(s)
I20260924 00:56:19.479007 12276 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
I20260924 00:56:19.487464 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 12276
I20260924 00:56:19.487668 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance
I20260924 00:56:19.488095 8450 external_mini_cluster.cc:1428] Setting key 25e350ece0f51dab36464ce4ee0fac0c
Sep 24 00:56:19 dist-test-slave-mmhs krb5kdc[12035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211379, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.130@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:56:19.499919 12272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:19.481712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:37773 (local address 127.8.64.190:46791)
0924 00:56:19.481898 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:19.481902 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:19.483096 (+ 1194us) server_negotiation.cc:407] Connection header received
0924 00:56:19.484891 (+ 1795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:19.484897 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:19.484985 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:19.485101 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:19.486585 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:19.487316 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:19.489152 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:19.489754 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:19.489902 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:19.493611 (+ 3709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:19.493617 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:19.493620 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:19.493652 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:19.496721 (+ 3069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:19.497481 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:19.497501 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:19.497503 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:19.497628 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:19.498095 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:19.498107 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:19.498110 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:19.498444 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:19.498624 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:19.499299 (+ 675us) server_negotiation.cc:299] Negotiation successful
0924 00:56:19.499538 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":78}
I20260924 00:56:19.500898 12406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:19.482111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:46791 (local address 127.8.64.130:37773)
0924 00:56:19.482890 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:19.482932 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:19.484620 (+ 1688us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:19.485331 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:19.485339 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:19.485843 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:19.486418 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:19.486435 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:19.487467 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:19.487479 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:19.488973 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:19.488992 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:19.489994 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:19.490004 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:19.490234 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:19.491027 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:19.491056 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:19.493358 (+ 2302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:19.496875 (+ 3517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:19.496885 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:19.496902 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:19.497344 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:19.497754 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:19.497761 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:19.497765 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:19.497964 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:19.498579 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:19.498590 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:19.498933 (+ 343us) client_negotiation.cc:769] Sending connection context
0924 00:56:19.499205 (+ 272us) client_negotiation.cc:240] Negotiation successful
0924 00:56:19.499746 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":361,"threads_started":1}
I20260924 00:56:19.502461 12404 heartbeater.cc:347] Connected to a master server at 127.8.64.190:46791
I20260924 00:56:19.502801 12404 heartbeater.cc:464] Registering TS with master...
I20260924 00:56:19.503695 12404 heartbeater.cc:511] Master 127.8.64.190:46791 requested a full tablet report, sending...
I20260924 00:56:19.505336 12080 ts_manager.cc:194] Registered new tserver with Master: f2a21b5a36cf4551b113db9618caa235 (127.8.64.130:37139)
I20260924 00:56:19.506142 12080 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.130@KRBTEST.COM'} at 127.8.64.130:37773
WARNING: no policy specified for kudu/127.8.64.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab.
Entry for principal kudu/127.8.64.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab.
I20260924 00:56:19.533891 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.131:0
--local_ip_for_outbound_sockets=127.8.64.131
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:46791
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab
--principal=kudu/127.8.64.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:40885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:56:19.848569 12409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:19.848848 12409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:19.848948 12409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:19.856118 12409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:19.856228 12409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:19.856338 12409 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.131
I20260924 00:56:19.865828 12409 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:40885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab
--principal=kudu/127.8.64.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:46791
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.131
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:19.867067 12409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:19.868770 12409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:19.881806 12414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:19.881747 12417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:19.881827 12415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:19.882651 12409 server_base.cc:1034] running on GCE node
I20260924 00:56:19.883358 12409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:19.884272 12409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:19.885504 12409 hybrid_clock.cc:648] HybridClock initialized: now 1790211379885466 us; error 54 us; skew 500 ppm
I20260924 00:56:19.885818 12409 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:19 dist-test-slave-mmhs krb5kdc[12035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211379, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:19.888962 12409 init.cc:377] Logged in from keytab as kudu/127.8.64.131@KRBTEST.COM (short username kudu)
I20260924 00:56:19.890597 12409 webserver.cc:460] Webserver started at http://127.8.64.131:33831/ using document root <none> and password file <none>
I20260924 00:56:19.891266 12409 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:19.891407 12409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:19.891707 12409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:56:19.894650 12409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-2/data/instance:
uuid: "65c0f876fad1441ea8ee7f74e64f1733"
format_stamp: "Formatted at 2026-09-24 00:56:19 on dist-test-slave-mmhs"
server_key: "e242687d215c440bd271a282780a270c"
server_key_iv: "f6aad1954aaac528006758f9e0f507b2"
server_key_version: "encryptionkey@0"
I20260924 00:56:19.895479 12409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance:
uuid: "65c0f876fad1441ea8ee7f74e64f1733"
format_stamp: "Formatted at 2026-09-24 00:56:19 on dist-test-slave-mmhs"
server_key: "e242687d215c440bd271a282780a270c"
server_key_iv: "f6aad1954aaac528006758f9e0f507b2"
server_key_version: "encryptionkey@0"
I20260924 00:56:19.900573 12409 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260924 00:56:19.904199 12424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:19.905515 12409 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260924 00:56:19.905692 12409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-2/wal
uuid: "65c0f876fad1441ea8ee7f74e64f1733"
format_stamp: "Formatted at 2026-09-24 00:56:19 on dist-test-slave-mmhs"
server_key: "e242687d215c440bd271a282780a270c"
server_key_iv: "f6aad1954aaac528006758f9e0f507b2"
server_key_version: "encryptionkey@0"
I20260924 00:56:19.905896 12409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:19.929704 12409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:19.934387 12409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:19.934696 12409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:19.935608 12409 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:56:19.936877 12409 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:56:19.936952 12409 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:19.937053 12409 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:56:19.937098 12409 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:19.969416 12409 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.131:38443
I20260924 00:56:19.969501 12536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.131:38443 every 8 connection(s)
I20260924 00:56:19.970695 12409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
I20260924 00:56:19.972079 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 12409
I20260924 00:56:19.972251 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance
I20260924 00:56:19.972602 8450 external_mini_cluster.cc:1428] Setting key c86842570b766e21f85b88a852200d26
Sep 24 00:56:19 dist-test-slave-mmhs krb5kdc[12035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211379, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.131@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:56:19.991720 12272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:19.974533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:56483 (local address 127.8.64.190:46791)
0924 00:56:19.974733 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:19.974739 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:19.976019 (+ 1280us) server_negotiation.cc:407] Connection header received
0924 00:56:19.977905 (+ 1886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:19.977910 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:19.978028 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:19.978139 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:19.979589 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:19.980719 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:19.982401 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:19.982780 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:19.982875 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:19.986106 (+ 3231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:19.986110 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:19.986114 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:19.986145 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:19.988797 (+ 2652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:19.989493 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:19.989500 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:19.989503 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:19.989628 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:19.990041 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:19.990048 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:19.990051 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:19.990385 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:19.990531 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:19.991155 (+ 624us) server_negotiation.cc:299] Negotiation successful
0924 00:56:19.991364 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260924 00:56:19.992709 12539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:19.975064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:46791 (local address 127.8.64.131:56483)
0924 00:56:19.975860 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:19.975917 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:19.977652 (+ 1735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:19.978417 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:19.978425 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:19.978892 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:19.979426 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:19.979441 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:19.980893 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:19.980898 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:19.982224 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:19.982236 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:19.982965 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:19.982969 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:19.983181 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:19.983791 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:19.983813 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:19.985886 (+ 2073us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:19.988936 (+ 3050us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:19.988945 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:19.988960 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:19.989374 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:19.989743 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:19.989748 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:19.989752 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:19.989932 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:19.990503 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:19.990511 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:19.990825 (+ 314us) client_negotiation.cc:769] Sending connection context
0924 00:56:19.991084 (+ 259us) client_negotiation.cc:240] Negotiation successful
0924 00:56:19.991588 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":339,"threads_started":1}
I20260924 00:56:19.994099 12537 heartbeater.cc:347] Connected to a master server at 127.8.64.190:46791
I20260924 00:56:19.994431 12537 heartbeater.cc:464] Registering TS with master...
I20260924 00:56:19.995132 12537 heartbeater.cc:511] Master 127.8.64.190:46791 requested a full tablet report, sending...
I20260924 00:56:19.996587 12080 ts_manager.cc:194] Registered new tserver with Master: 65c0f876fad1441ea8ee7f74e64f1733 (127.8.64.131:38443)
I20260924 00:56:19.997399 12080 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.131@KRBTEST.COM'} at 127.8.64.131:56483
I20260924 00:56:19.999989 8450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 00:56:20.010888 12270 heartbeater.cc:503] Master 127.8.64.190:46791 was elected leader, sending a full tablet report...
Sep 24 00:56:20 dist-test-slave-mmhs krb5kdc[12035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 24 00:56:20 dist-test-slave-mmhs krb5kdc[12035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.129@KRBTEST.COM
I20260924 00:56:20.048425 12544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:20.033199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54132 (local address 127.8.64.129:45987)
0924 00:56:20.033980 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:20.033987 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:20.034013 (+ 26us) server_negotiation.cc:407] Connection header received
0924 00:56:20.034080 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:20.034086 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:20.034379 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:20.034523 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:20.035886 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:20.036899 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:20.038062 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:20.038721 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:20.038819 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:20.042720 (+ 3901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:20.042732 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:20.042746 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:20.042785 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:20.045606 (+ 2821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:20.046277 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:20.046284 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:20.046287 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:20.046506 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:20.046923 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:20.046926 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:20.046929 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:20.047377 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:20.047591 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:20.047897 (+ 306us) server_negotiation.cc:299] Negotiation successful
0924 00:56:20.048124 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":648,"thread_start_us":476,"threads_started":1}
Sep 24 00:56:20 dist-test-slave-mmhs krb5kdc[12035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:56:20.066428 12272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:20.052549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51950 (local address 127.8.64.190:46791)
0924 00:56:20.052776 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:20.052783 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:20.052942 (+ 159us) server_negotiation.cc:407] Connection header received
0924 00:56:20.053124 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:20.053130 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:20.053204 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:20.053329 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:20.054579 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:20.055318 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:20.056543 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:20.056982 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:20.057110 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:20.060806 (+ 3696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:20.060812 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:20.060824 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:20.060855 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:20.063846 (+ 2991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:20.064459 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:20.064466 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:20.064470 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:20.064597 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:20.064972 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:20.064978 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:20.064981 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:20.065318 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:20.065559 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:20.065864 (+ 305us) server_negotiation.cc:299] Negotiation successful
0924 00:56:20.066064 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260924 00:56:20.067049 12080 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51950
I20260924 00:56:20.068099 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 12142
I20260924 00:56:20.080276 12544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:20.069051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54148 (local address 127.8.64.129:45987)
0924 00:56:20.069252 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:20.069259 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:20.069442 (+ 183us) server_negotiation.cc:407] Connection header received
0924 00:56:20.069640 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:20.069644 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:20.069705 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:20.069818 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:20.071025 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:20.071708 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:20.072901 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:20.073341 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:20.073478 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:20.075778 (+ 2300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:20.075783 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:20.075786 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:20.075815 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:20.078136 (+ 2321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:20.078746 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:20.078749 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:20.078752 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:20.078862 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:20.079158 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:20.079161 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:20.079163 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:20.079374 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:20.079535 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:20.079794 (+ 259us) server_negotiation.cc:299] Negotiation successful
0924 00:56:20.079940 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260924 00:56:20.080881 12265 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:56:20.163043 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 12142
I20260924 00:56:20.179665 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 12276
Sep 24 00:56:20 dist-test-slave-mmhs krb5kdc[12035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.130@KRBTEST.COM
I20260924 00:56:20.194766 12549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:20.180552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60970 (local address 127.8.64.130:37139)
0924 00:56:20.181128 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:20.181135 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:20.181162 (+ 27us) server_negotiation.cc:407] Connection header received
0924 00:56:20.181223 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:20.181230 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:20.181438 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:20.181575 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:20.182755 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:20.183760 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:20.184938 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:20.185665 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:20.185774 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:20.189691 (+ 3917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:20.189700 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:20.189726 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:20.189762 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:20.192341 (+ 2579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:20.192888 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:20.192894 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:20.192897 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:20.193011 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:20.193405 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:20.193408 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:20.193410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:20.193703 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:20.193921 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:20.194205 (+ 284us) server_negotiation.cc:299] Negotiation successful
0924 00:56:20.194481 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":332,"threads_started":1}
I20260924 00:56:20.196056 12399 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:56:20.276932 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 12276
I20260924 00:56:20.292868 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 12409
Sep 24 00:56:20 dist-test-slave-mmhs krb5kdc[12035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.131@KRBTEST.COM
I20260924 00:56:20.308614 12551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:20.294022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36486 (local address 127.8.64.131:38443)
0924 00:56:20.294743 (+ 721us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:20.294750 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:20.294776 (+ 26us) server_negotiation.cc:407] Connection header received
0924 00:56:20.294833 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:20.294840 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:20.295044 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:20.295173 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:20.296359 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:20.297316 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:20.298708 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:20.299232 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:20.299324 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:20.303395 (+ 4071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:20.303405 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:20.303418 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:20.303452 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:20.306186 (+ 2734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:20.306855 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:20.306861 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:20.306864 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:20.306974 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:20.307342 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:20.307345 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:20.307346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:20.307662 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:20.307871 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:20.308143 (+ 272us) server_negotiation.cc:299] Negotiation successful
0924 00:56:20.308339 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":417,"threads_started":1}
I20260924 00:56:20.309963 12532 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:56:20.392069 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 12409
I20260924 00:56:20.410190 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 12048
I20260924 00:56:20.411339 12110 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:56:20.512454 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 12048
2026-09-24T00:56:20Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3858 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:56:20 dist-test-slave-mmhs krb5kdc[12556](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:56:20 dist-test-slave-mmhs krb5kdc[12556](info): set up 2 sockets
Sep 24 00:56:20 dist-test-slave-mmhs krb5kdc[12556](info): commencing operation
krb5kdc: starting...
W20260924 00:56:21.640424 8450 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.001s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Sep 24 00:56:21 dist-test-slave-mmhs krb5kdc[12556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:56:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:56:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.64.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
I20260924 00:56:21.841545 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:43917
--webserver_interface=127.8.64.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:38689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:43917
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:56:22.154911 12569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:22.155262 12569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:22.155433 12569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:22.162431 12569 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:56:22.162585 12569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:22.162695 12569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:22.162801 12569 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:56:22.162896 12569 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:56:22.172361 12569 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:38689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:43917
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:43917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:22.173609 12569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:22.174916 12569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:22.186901 12574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:22.186836 12575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:22.186959 12577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:22.187685 12569 server_base.cc:1034] running on GCE node
I20260924 00:56:22.188608 12569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:22.190050 12569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:22.191256 12569 hybrid_clock.cc:648] HybridClock initialized: now 1790211382191192 us; error 62 us; skew 500 ppm
I20260924 00:56:22.191603 12569 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:22 dist-test-slave-mmhs krb5kdc[12556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211382, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:22.194629 12569 init.cc:377] Logged in from keytab as kudu/127.8.64.190@KRBTEST.COM (short username kudu)
I20260924 00:56:22.195766 12569 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:22.195851 12569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:22.196171 12569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:56:22.199115 12569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "6517639d582a4c82aa9ea469e549beee"
format_stamp: "Formatted at 2026-09-24 00:56:22 on dist-test-slave-mmhs"
server_key: "8157c96d3dfb8e2b729be473c83d2de9"
server_key_iv: "a9891573abe427172e5669822f743586"
server_key_version: "encryptionkey@0"
I20260924 00:56:22.199873 12569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "6517639d582a4c82aa9ea469e549beee"
format_stamp: "Formatted at 2026-09-24 00:56:22 on dist-test-slave-mmhs"
server_key: "8157c96d3dfb8e2b729be473c83d2de9"
server_key_iv: "a9891573abe427172e5669822f743586"
server_key_version: "encryptionkey@0"
I20260924 00:56:22.204610 12569 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260924 00:56:22.207903 12583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:22.209105 12569 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260924 00:56:22.209282 12569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "6517639d582a4c82aa9ea469e549beee"
format_stamp: "Formatted at 2026-09-24 00:56:22 on dist-test-slave-mmhs"
server_key: "8157c96d3dfb8e2b729be473c83d2de9"
server_key_iv: "a9891573abe427172e5669822f743586"
server_key_version: "encryptionkey@0"
I20260924 00:56:22.209491 12569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:22.224587 12569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:22.228480 12569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:22.228752 12569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:22.247591 12569 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.190:43917
I20260924 00:56:22.247611 12634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.190:43917 every 8 connection(s)
I20260924 00:56:22.248508 12569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 00:56:22.249958 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 12569
I20260924 00:56:22.250197 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 00:56:22.250824 8450 external_mini_cluster.cc:1428] Setting key ab7de34717d1a40158b1ce59e21707c3
I20260924 00:56:22.252859 12635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 24 00:56:22 dist-test-slave-mmhs krb5kdc[12556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:56:22.267174 12635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6517639d582a4c82aa9ea469e549beee: Bootstrap starting.
I20260924 00:56:22.270836 12635 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6517639d582a4c82aa9ea469e549beee: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:22.272234 12635 log.cc:824] T 00000000000000000000000000000000 P 6517639d582a4c82aa9ea469e549beee: Log is configured to *not* fsync() on all Append() calls
I20260924 00:56:22.274459 12638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:22.252743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58392 (local address 127.8.64.190:43917)
0924 00:56:22.253590 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:22.253601 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:22.253640 (+ 39us) server_negotiation.cc:407] Connection header received
0924 00:56:22.255274 (+ 1634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:22.255295 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:22.255674 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:22.255996 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:22.257971 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:22.259359 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:22.261325 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:22.261945 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:22.262064 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:22.266874 (+ 4810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:22.266881 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:22.266895 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:22.266927 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:22.269863 (+ 2936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:22.270597 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:22.270608 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:22.270620 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:22.270866 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:22.271256 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:22.271264 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:22.271273 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:22.272000 (+ 727us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:22.272344 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:22.273152 (+ 808us) server_negotiation.cc:299] Negotiation successful
0924 00:56:22.273466 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":428,"threads_started":1}
I20260924 00:56:22.275279 12635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6517639d582a4c82aa9ea469e549beee: No bootstrap required, opened a new log
I20260924 00:56:22.282946 12635 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6517639d582a4c82aa9ea469e549beee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6517639d582a4c82aa9ea469e549beee" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 43917 } }
I20260924 00:56:22.283344 12635 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6517639d582a4c82aa9ea469e549beee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:22.283463 12635 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6517639d582a4c82aa9ea469e549beee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6517639d582a4c82aa9ea469e549beee, State: Initialized, Role: FOLLOWER
I20260924 00:56:22.284013 12635 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6517639d582a4c82aa9ea469e549beee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6517639d582a4c82aa9ea469e549beee" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 43917 } }
I20260924 00:56:22.284274 12635 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6517639d582a4c82aa9ea469e549beee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:56:22.284423 12635 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6517639d582a4c82aa9ea469e549beee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:56:22.284616 12635 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6517639d582a4c82aa9ea469e549beee [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:22.287014 12635 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6517639d582a4c82aa9ea469e549beee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6517639d582a4c82aa9ea469e549beee" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 43917 } }
I20260924 00:56:22.287518 12635 leader_election.cc:302] T 00000000000000000000000000000000 P 6517639d582a4c82aa9ea469e549beee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6517639d582a4c82aa9ea469e549beee; no voters:
I20260924 00:56:22.288091 12635 leader_election.cc:288] T 00000000000000000000000000000000 P 6517639d582a4c82aa9ea469e549beee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:56:22.288234 12640 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6517639d582a4c82aa9ea469e549beee [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:56:22.288961 12640 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6517639d582a4c82aa9ea469e549beee [term 1 LEADER]: Becoming Leader. State: Replica: 6517639d582a4c82aa9ea469e549beee, State: Running, Role: LEADER
I20260924 00:56:22.289470 12640 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6517639d582a4c82aa9ea469e549beee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6517639d582a4c82aa9ea469e549beee" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 43917 } }
I20260924 00:56:22.290278 12635 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6517639d582a4c82aa9ea469e549beee [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:56:22.292359 12642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6517639d582a4c82aa9ea469e549beee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6517639d582a4c82aa9ea469e549beee. Latest consensus state: current_term: 1 leader_uuid: "6517639d582a4c82aa9ea469e549beee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6517639d582a4c82aa9ea469e549beee" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 43917 } } }
I20260924 00:56:22.292840 12642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6517639d582a4c82aa9ea469e549beee [sys.catalog]: This master's current role is: LEADER
I20260924 00:56:22.293880 12649 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:56:22.293751 12641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6517639d582a4c82aa9ea469e549beee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6517639d582a4c82aa9ea469e549beee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6517639d582a4c82aa9ea469e549beee" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 43917 } } }
I20260924 00:56:22.294209 12641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6517639d582a4c82aa9ea469e549beee [sys.catalog]: This master's current role is: LEADER
I20260924 00:56:22.304294 12649 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:56:22.312390 12649 catalog_manager.cc:1347] Generated new cluster ID: 747cc7f2c8e343ffbbd3e7904a2e4007
I20260924 00:56:22.312503 12649 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:56:22.320216 12649 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:56:22.321393 12649 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:56:22.333192 12649 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6517639d582a4c82aa9ea469e549beee: Generated new TSK 0
I20260924 00:56:22.334036 12649 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.64.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab.
Entry for principal kudu/127.8.64.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab.
I20260924 00:56:22.398804 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.129:0
--local_ip_for_outbound_sockets=127.8.64.129
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:43917
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab
--principal=kudu/127.8.64.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:38689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:56:22.729031 12661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:22.729314 12661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:22.729478 12661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:22.736869 12661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:22.737026 12661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:22.737232 12661 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.129
I20260924 00:56:22.746954 12661 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:38689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab
--principal=kudu/127.8.64.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:43917
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.129
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:22.748310 12661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:22.749647 12661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:22.762804 12666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:22.762830 12667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:22.762887 12669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:22.763819 12661 server_base.cc:1034] running on GCE node
I20260924 00:56:22.764386 12661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:22.765285 12661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:22.766472 12661 hybrid_clock.cc:648] HybridClock initialized: now 1790211382766422 us; error 40 us; skew 500 ppm
I20260924 00:56:22.766793 12661 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:22 dist-test-slave-mmhs krb5kdc[12556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211382, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:22.769863 12661 init.cc:377] Logged in from keytab as kudu/127.8.64.129@KRBTEST.COM (short username kudu)
I20260924 00:56:22.770954 12661 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:22.771061 12661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:22.771368 12661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:56:22.774261 12661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-0/data/instance:
uuid: "5c78d6525d0e4f35949b848e3b2bc92c"
format_stamp: "Formatted at 2026-09-24 00:56:22 on dist-test-slave-mmhs"
server_key: "5ffa1a973e23d13331b3155908b80dd7"
server_key_iv: "2a08271147e960943a6bac9cf01d06d4"
server_key_version: "encryptionkey@0"
I20260924 00:56:22.775130 12661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance:
uuid: "5c78d6525d0e4f35949b848e3b2bc92c"
format_stamp: "Formatted at 2026-09-24 00:56:22 on dist-test-slave-mmhs"
server_key: "5ffa1a973e23d13331b3155908b80dd7"
server_key_iv: "2a08271147e960943a6bac9cf01d06d4"
server_key_version: "encryptionkey@0"
I20260924 00:56:22.779934 12661 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260924 00:56:22.783336 12675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:22.784588 12661 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260924 00:56:22.784759 12661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-0/wal
uuid: "5c78d6525d0e4f35949b848e3b2bc92c"
format_stamp: "Formatted at 2026-09-24 00:56:22 on dist-test-slave-mmhs"
server_key: "5ffa1a973e23d13331b3155908b80dd7"
server_key_iv: "2a08271147e960943a6bac9cf01d06d4"
server_key_version: "encryptionkey@0"
I20260924 00:56:22.784971 12661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:22.816025 12661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:22.819906 12661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:22.820178 12661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:22.821128 12661 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:56:22.822403 12661 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:56:22.822485 12661 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:22.822562 12661 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:56:22.822620 12661 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:22.855443 12661 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.129:34959
I20260924 00:56:22.855526 12787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.129:34959 every 8 connection(s)
I20260924 00:56:22.856297 12661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
I20260924 00:56:22.864444 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 12661
I20260924 00:56:22.864650 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance
I20260924 00:56:22.865253 8450 external_mini_cluster.cc:1428] Setting key 75d030bd1409fb191b993f73229227fd
Sep 24 00:56:22 dist-test-slave-mmhs krb5kdc[12556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211382, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:56:22.875135 12790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:22.858641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:41561 (local address 127.8.64.190:43917)
0924 00:56:22.859095 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:22.859099 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:22.859926 (+ 827us) server_negotiation.cc:407] Connection header received
0924 00:56:22.861752 (+ 1826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:22.861758 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:22.861846 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:22.861962 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:22.863407 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:22.864096 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:22.865782 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:22.866373 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:22.866505 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:22.869779 (+ 3274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:22.869785 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:22.869790 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:22.869829 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:22.872220 (+ 2391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:22.872920 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:22.872928 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:22.872931 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:22.873049 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:22.873452 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:22.873458 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:22.873462 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:22.873712 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:22.873861 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:22.874567 (+ 706us) server_negotiation.cc:299] Negotiation successful
0924 00:56:22.874779 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":238,"threads_started":1}
I20260924 00:56:22.876091 12791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:22.859106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:43917 (local address 127.8.64.129:41561)
0924 00:56:22.859769 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:22.859805 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:22.861492 (+ 1687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:22.862193 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:22.862201 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:22.862706 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:22.863231 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:22.863247 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:22.864254 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:22.864260 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:22.865628 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:22.865639 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:22.866579 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:22.866587 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:22.866823 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:22.867528 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:22.867549 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:22.869574 (+ 2025us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:22.872360 (+ 2786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:22.872369 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:22.872384 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:22.872801 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:22.873157 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:22.873163 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:22.873167 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:22.873343 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:22.873834 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:22.873841 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:22.874161 (+ 320us) client_negotiation.cc:769] Sending connection context
0924 00:56:22.874475 (+ 314us) client_negotiation.cc:240] Negotiation successful
0924 00:56:22.874978 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":272,"threads_started":1}
I20260924 00:56:22.877637 12788 heartbeater.cc:347] Connected to a master server at 127.8.64.190:43917
I20260924 00:56:22.877928 12788 heartbeater.cc:464] Registering TS with master...
I20260924 00:56:22.878592 12788 heartbeater.cc:511] Master 127.8.64.190:43917 requested a full tablet report, sending...
I20260924 00:56:22.880558 12600 ts_manager.cc:194] Registered new tserver with Master: 5c78d6525d0e4f35949b848e3b2bc92c (127.8.64.129:34959)
I20260924 00:56:22.881963 12600 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.129@KRBTEST.COM'} at 127.8.64.129:41561
WARNING: no policy specified for kudu/127.8.64.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab.
Entry for principal kudu/127.8.64.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab.
I20260924 00:56:22.910046 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.130:0
--local_ip_for_outbound_sockets=127.8.64.130
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:43917
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab
--principal=kudu/127.8.64.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:38689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:56:23.231786 12794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:23.232084 12794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:23.232259 12794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:23.239621 12794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:23.239766 12794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:23.239953 12794 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.130
I20260924 00:56:23.251315 12794 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:38689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab
--principal=kudu/127.8.64.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:43917
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.130
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:23.252789 12794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:23.254433 12794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:23.268494 12799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:23.268595 12802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:23.268649 12800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:23.269150 12794 server_base.cc:1034] running on GCE node
I20260924 00:56:23.269721 12794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:23.270718 12794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:23.271944 12794 hybrid_clock.cc:648] HybridClock initialized: now 1790211383271885 us; error 57 us; skew 500 ppm
I20260924 00:56:23.272254 12794 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:23 dist-test-slave-mmhs krb5kdc[12556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211383, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:23.275444 12794 init.cc:377] Logged in from keytab as kudu/127.8.64.130@KRBTEST.COM (short username kudu)
I20260924 00:56:23.276628 12794 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:23.276762 12794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:23.277070 12794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:56:23.279960 12794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-1/data/instance:
uuid: "a8ac3fccf4864177bba425c2fc10877d"
format_stamp: "Formatted at 2026-09-24 00:56:23 on dist-test-slave-mmhs"
server_key: "9622eccec1ad3281be9a25496f4090de"
server_key_iv: "98d6ee19add237ce24ef217ebe970be6"
server_key_version: "encryptionkey@0"
I20260924 00:56:23.280754 12794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance:
uuid: "a8ac3fccf4864177bba425c2fc10877d"
format_stamp: "Formatted at 2026-09-24 00:56:23 on dist-test-slave-mmhs"
server_key: "9622eccec1ad3281be9a25496f4090de"
server_key_iv: "98d6ee19add237ce24ef217ebe970be6"
server_key_version: "encryptionkey@0"
I20260924 00:56:23.285501 12794 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260924 00:56:23.288925 12808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:23.290436 12794 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 00:56:23.290606 12794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-1/wal
uuid: "a8ac3fccf4864177bba425c2fc10877d"
format_stamp: "Formatted at 2026-09-24 00:56:23 on dist-test-slave-mmhs"
server_key: "9622eccec1ad3281be9a25496f4090de"
server_key_iv: "98d6ee19add237ce24ef217ebe970be6"
server_key_version: "encryptionkey@0"
I20260924 00:56:23.290831 12794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:23.311887 12794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:23.316128 12794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:23.316398 12794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:23.317281 12794 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:56:23.318699 12794 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:56:23.318845 12794 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:23.318981 12794 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:56:23.319118 12794 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:23.351807 12794 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.130:44225
I20260924 00:56:23.351850 12920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.130:44225 every 8 connection(s)
I20260924 00:56:23.352674 12794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
I20260924 00:56:23.358541 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 12794
I20260924 00:56:23.358764 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance
I20260924 00:56:23.359220 8450 external_mini_cluster.cc:1428] Setting key bc08c6e4eb8718ab94b00f63456abaf4
Sep 24 00:56:23 dist-test-slave-mmhs krb5kdc[12556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211383, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.130@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.64.131@KRBTEST.COM; defaulting to no policy
I20260924 00:56:23.379590 12790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:23.355092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:48587 (local address 127.8.64.190:43917)
0924 00:56:23.355309 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:23.355315 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:23.356449 (+ 1134us) server_negotiation.cc:407] Connection header received
0924 00:56:23.358278 (+ 1829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:23.358286 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:23.358402 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:23.358524 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:23.360183 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:23.361031 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:23.362559 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:23.363326 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:23.363468 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:23.373054 (+ 9586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:23.373059 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:23.373063 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:23.373128 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:23.376635 (+ 3507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:23.377363 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:23.377369 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:23.377373 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:23.377527 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:23.377940 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:23.377947 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:23.377952 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:23.378286 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:23.378472 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:23.379134 (+ 662us) server_negotiation.cc:299] Negotiation successful
0924 00:56:23.379299 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260924 00:56:23.380652 12923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:23.355595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:43917 (local address 127.8.64.130:48587)
0924 00:56:23.356296 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:23.356333 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:23.358006 (+ 1673us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:23.358751 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:23.358760 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:23.359336 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:23.360010 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:23.360029 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:23.361169 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:23.361173 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:23.362423 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:23.362435 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:23.363519 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:23.363523 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:23.363701 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:23.370138 (+ 6437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:23.370172 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:23.372816 (+ 2644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:23.376796 (+ 3980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:23.376803 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:23.376817 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:23.377238 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:23.377646 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:23.377649 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:23.377652 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:23.377821 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:23.378453 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:23.378462 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:23.378809 (+ 347us) client_negotiation.cc:769] Sending connection context
0924 00:56:23.379038 (+ 229us) client_negotiation.cc:240] Negotiation successful
0924 00:56:23.379418 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":291,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.131@KRBTEST.COM" created.
I20260924 00:56:23.382097 12921 heartbeater.cc:347] Connected to a master server at 127.8.64.190:43917
I20260924 00:56:23.382448 12921 heartbeater.cc:464] Registering TS with master...
I20260924 00:56:23.383250 12921 heartbeater.cc:511] Master 127.8.64.190:43917 requested a full tablet report, sending...
I20260924 00:56:23.384810 12600 ts_manager.cc:194] Registered new tserver with Master: a8ac3fccf4864177bba425c2fc10877d (127.8.64.130:44225)
I20260924 00:56:23.385581 12600 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.130@KRBTEST.COM'} at 127.8.64.130:48587
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab.
Entry for principal kudu/127.8.64.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab.
I20260924 00:56:23.409003 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.131:0
--local_ip_for_outbound_sockets=127.8.64.131
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:43917
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab
--principal=kudu/127.8.64.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:38689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:56:23.727341 12926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:23.727638 12926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:23.727811 12926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:23.735035 12926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:23.735173 12926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:23.735420 12926 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.131
I20260924 00:56:23.744978 12926 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:38689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab
--principal=kudu/127.8.64.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:43917
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.131
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:23.746222 12926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:23.747510 12926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:23.759786 12931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:23.759863 12934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:23.759895 12932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:23.760879 12926 server_base.cc:1034] running on GCE node
I20260924 00:56:23.761427 12926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:23.762240 12926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:23.763473 12926 hybrid_clock.cc:648] HybridClock initialized: now 1790211383763433 us; error 37 us; skew 500 ppm
I20260924 00:56:23.763775 12926 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:23 dist-test-slave-mmhs krb5kdc[12556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211383, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:23.766734 12926 init.cc:377] Logged in from keytab as kudu/127.8.64.131@KRBTEST.COM (short username kudu)
I20260924 00:56:23.767798 12926 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:23.767927 12926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:23.768239 12926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:56:23.771093 12926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-2/data/instance:
uuid: "cd2c7168c6124502af2f361fabd13ef4"
format_stamp: "Formatted at 2026-09-24 00:56:23 on dist-test-slave-mmhs"
server_key: "3e42b6d27e9aecb2792212ab3d7f9ba8"
server_key_iv: "23dd858ece6fa8cedb1940453b0942e6"
server_key_version: "encryptionkey@0"
I20260924 00:56:23.771900 12926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance:
uuid: "cd2c7168c6124502af2f361fabd13ef4"
format_stamp: "Formatted at 2026-09-24 00:56:23 on dist-test-slave-mmhs"
server_key: "3e42b6d27e9aecb2792212ab3d7f9ba8"
server_key_iv: "23dd858ece6fa8cedb1940453b0942e6"
server_key_version: "encryptionkey@0"
I20260924 00:56:23.776928 12926 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260924 00:56:23.780334 12940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:23.781721 12926 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260924 00:56:23.781915 12926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-2/wal
uuid: "cd2c7168c6124502af2f361fabd13ef4"
format_stamp: "Formatted at 2026-09-24 00:56:23 on dist-test-slave-mmhs"
server_key: "3e42b6d27e9aecb2792212ab3d7f9ba8"
server_key_iv: "23dd858ece6fa8cedb1940453b0942e6"
server_key_version: "encryptionkey@0"
I20260924 00:56:23.782135 12926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:23.795017 12926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:23.799293 12926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:23.799505 12926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:23.800336 12926 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:56:23.801522 12926 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:56:23.801600 12926 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:23.801673 12926 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:56:23.801726 12926 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:23.832978 12926 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.131:43947
I20260924 00:56:23.833060 13052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.131:43947 every 8 connection(s)
I20260924 00:56:23.833833 12926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
I20260924 00:56:23.837517 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 12926
I20260924 00:56:23.837733 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance
I20260924 00:56:23.838201 8450 external_mini_cluster.cc:1428] Setting key 14689cf854b0c698530838811755b182
Sep 24 00:56:23 dist-test-slave-mmhs krb5kdc[12556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211383, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.131@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:56:23.853008 12790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:23.836319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:38555 (local address 127.8.64.190:43917)
0924 00:56:23.836490 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:23.836495 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:23.837652 (+ 1157us) server_negotiation.cc:407] Connection header received
0924 00:56:23.839743 (+ 2091us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:23.839748 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:23.839832 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:23.839941 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:23.841347 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:23.842049 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:23.843614 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:23.844010 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:23.844146 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:23.847513 (+ 3367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:23.847518 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:23.847521 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:23.847551 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:23.849921 (+ 2370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:23.850705 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:23.850711 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:23.850714 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:23.850832 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:23.851251 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:23.851255 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:23.851259 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:23.851512 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:23.851668 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:23.852411 (+ 743us) server_negotiation.cc:299] Negotiation successful
0924 00:56:23.852625 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260924 00:56:23.853917 13055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:23.836805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:43917 (local address 127.8.64.131:38555)
0924 00:56:23.837479 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:23.837520 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:23.839454 (+ 1934us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:23.840163 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:23.840171 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:23.840648 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:23.841175 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:23.841190 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:23.842197 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:23.842204 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:23.843464 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:23.843475 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:23.844234 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:23.844241 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:23.844482 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:23.845122 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:23.845147 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:23.847306 (+ 2159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:23.850058 (+ 2752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:23.850066 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:23.850081 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:23.850582 (+ 501us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:23.850950 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:23.850955 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:23.850958 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:23.851124 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:23.851636 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:23.851644 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:23.851966 (+ 322us) client_negotiation.cc:769] Sending connection context
0924 00:56:23.852261 (+ 295us) client_negotiation.cc:240] Negotiation successful
0924 00:56:23.852716 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":304,"threads_started":1}
I20260924 00:56:23.855417 13053 heartbeater.cc:347] Connected to a master server at 127.8.64.190:43917
I20260924 00:56:23.855715 13053 heartbeater.cc:464] Registering TS with master...
I20260924 00:56:23.856362 13053 heartbeater.cc:511] Master 127.8.64.190:43917 requested a full tablet report, sending...
I20260924 00:56:23.857844 12600 ts_manager.cc:194] Registered new tserver with Master: cd2c7168c6124502af2f361fabd13ef4 (127.8.64.131:43947)
I20260924 00:56:23.858959 12600 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.131@KRBTEST.COM'} at 127.8.64.131:38555
I20260924 00:56:23.865442 8450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 00:56:23.881163 12790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:23.869651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58394 (local address 127.8.64.190:43917)
0924 00:56:23.869805 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:23.869810 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:23.870382 (+ 572us) server_negotiation.cc:407] Connection header received
0924 00:56:23.870526 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:23.870530 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:23.870585 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:23.870668 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:23.872022 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:23.872736 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:23.873906 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:23.874361 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:23.874452 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:23.876513 (+ 2061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:23.876518 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:23.876521 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:23.876547 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:23.879041 (+ 2494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:23.879600 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:23.879604 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:23.879606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:23.879714 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:23.880005 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:23.880008 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:23.880010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:23.880328 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:23.880498 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:23.880762 (+ 264us) server_negotiation.cc:299] Negotiation successful
0924 00:56:23.880905 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260924 00:56:23.884802 12788 heartbeater.cc:503] Master 127.8.64.190:43917 was elected leader, sending a full tablet report...
I20260924 00:56:23.885183 12600 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58394:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:56:23.892688 12600 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:56:23.919267 13070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:23.905961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47760 (local address 127.8.64.129:34959)
0924 00:56:23.912432 (+ 6471us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:23.912440 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:23.912464 (+ 24us) server_negotiation.cc:407] Connection header received
0924 00:56:23.912553 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:23.912562 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:23.912773 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:23.912957 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:56:23.913524 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:23.914919 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:23.917107 (+ 2188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:23.918025 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:23.918234 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:23.918510 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:23.918740 (+ 230us) server_negotiation.cc:299] Negotiation successful
0924 00:56:23.918921 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6259,"thread_start_us":6348,"threads_started":1}
I20260924 00:56:23.919317 13069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:23.905410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:34959 (local address 127.0.0.1:47760)
0924 00:56:23.906003 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:23.906025 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:23.906155 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:23.912953 (+ 6798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:23.912957 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:23.912991 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:56:23.913336 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:23.913344 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:23.915058 (+ 1714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:23.915063 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:23.916956 (+ 1893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:23.916968 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:23.918283 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:23.918289 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:23.918504 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:23.918552 (+ 48us) client_negotiation.cc:769] Sending connection context
0924 00:56:23.918690 (+ 138us) client_negotiation.cc:240] Negotiation successful
0924 00:56:23.918884 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"mutex_wait_us":1,"thread_start_us":689,"threads_started":1}
I20260924 00:56:23.920668 13065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:23.903792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.130:44225 (local address 127.0.0.1:39932)
0924 00:56:23.904648 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:23.904669 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:23.904841 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:23.912508 (+ 7667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:23.912513 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:23.912536 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:56:23.912934 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:23.912946 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:23.916026 (+ 3080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:23.916031 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:23.917737 (+ 1706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:23.917749 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:23.919786 (+ 2037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:23.919792 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:23.919946 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:23.919976 (+ 30us) client_negotiation.cc:769] Sending connection context
0924 00:56:23.920097 (+ 121us) client_negotiation.cc:240] Negotiation successful
0924 00:56:23.920231 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":691,"mutex_wait_us":20,"spinlock_wait_cycles":1408,"thread_start_us":1223,"threads_started":1}
I20260924 00:56:23.920707 13067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:23.904136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39932 (local address 127.8.64.130:44225)
0924 00:56:23.912003 (+ 7867us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:23.912012 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:23.912037 (+ 25us) server_negotiation.cc:407] Connection header received
0924 00:56:23.912119 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:23.912124 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:23.912329 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:23.912517 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:56:23.914394 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:23.915878 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:23.917889 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:23.919346 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:23.919732 (+ 386us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:23.919877 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:23.920200 (+ 323us) server_negotiation.cc:299] Negotiation successful
0924 00:56:23.920369 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7709,"thread_start_us":7592,"threads_started":1}
I20260924 00:56:23.922418 13068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:23.904303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58884 (local address 127.8.64.131:43947)
0924 00:56:23.910817 (+ 6514us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:23.910824 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:23.910850 (+ 26us) server_negotiation.cc:407] Connection header received
0924 00:56:23.910924 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:23.910930 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:23.911168 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:23.911394 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:56:23.912887 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:23.914575 (+ 1688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:23.918154 (+ 3579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:23.921297 (+ 3143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:23.921450 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:23.921677 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:23.921896 (+ 219us) server_negotiation.cc:299] Negotiation successful
0924 00:56:23.922049 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6363,"thread_start_us":6208,"threads_started":1}
I20260924 00:56:23.922433 13066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:23.903768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.131:43947 (local address 127.0.0.1:58884)
0924 00:56:23.904621 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:23.904669 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:23.904840 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:23.912012 (+ 7172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:23.912023 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:23.912060 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:56:23.912639 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:23.912657 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:23.914707 (+ 2050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:23.914712 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:23.917995 (+ 3283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:23.918008 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:23.921493 (+ 3485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:23.921499 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:23.921672 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:23.921711 (+ 39us) client_negotiation.cc:769] Sending connection context
0924 00:56:23.921827 (+ 116us) client_negotiation.cc:240] Negotiation successful
0924 00:56:23.922015 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"mutex_wait_us":4,"spinlock_wait_cycles":6400,"thread_start_us":583,"threads_started":1}
I20260924 00:56:23.923429 12723 tablet_service.cc:1467] Processing CreateTablet for tablet 92912b67deb84581841986636ac20315 (DEFAULT_TABLE table=test-table [id=10332cedd53745c38c655fe821b0bd6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:56:23.924389 12856 tablet_service.cc:1467] Processing CreateTablet for tablet 92912b67deb84581841986636ac20315 (DEFAULT_TABLE table=test-table [id=10332cedd53745c38c655fe821b0bd6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:56:23.924853 12723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92912b67deb84581841986636ac20315. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:23.925478 12856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92912b67deb84581841986636ac20315. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:23.926267 12988 tablet_service.cc:1467] Processing CreateTablet for tablet 92912b67deb84581841986636ac20315 (DEFAULT_TABLE table=test-table [id=10332cedd53745c38c655fe821b0bd6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:56:23.927490 12988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92912b67deb84581841986636ac20315. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:23.937117 13071 tablet_bootstrap.cc:492] T 92912b67deb84581841986636ac20315 P a8ac3fccf4864177bba425c2fc10877d: Bootstrap starting.
I20260924 00:56:23.940374 13071 tablet_bootstrap.cc:654] T 92912b67deb84581841986636ac20315 P a8ac3fccf4864177bba425c2fc10877d: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:23.942603 13073 tablet_bootstrap.cc:492] T 92912b67deb84581841986636ac20315 P cd2c7168c6124502af2f361fabd13ef4: Bootstrap starting.
I20260924 00:56:23.943266 13071 log.cc:824] T 92912b67deb84581841986636ac20315 P a8ac3fccf4864177bba425c2fc10877d: Log is configured to *not* fsync() on all Append() calls
I20260924 00:56:23.944191 13072 tablet_bootstrap.cc:492] T 92912b67deb84581841986636ac20315 P 5c78d6525d0e4f35949b848e3b2bc92c: Bootstrap starting.
I20260924 00:56:23.946494 13073 tablet_bootstrap.cc:654] T 92912b67deb84581841986636ac20315 P cd2c7168c6124502af2f361fabd13ef4: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:23.947223 13072 tablet_bootstrap.cc:654] T 92912b67deb84581841986636ac20315 P 5c78d6525d0e4f35949b848e3b2bc92c: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:23.947729 13071 tablet_bootstrap.cc:492] T 92912b67deb84581841986636ac20315 P a8ac3fccf4864177bba425c2fc10877d: No bootstrap required, opened a new log
I20260924 00:56:23.947779 13073 log.cc:824] T 92912b67deb84581841986636ac20315 P cd2c7168c6124502af2f361fabd13ef4: Log is configured to *not* fsync() on all Append() calls
I20260924 00:56:23.948017 13071 ts_tablet_manager.cc:1397] T 92912b67deb84581841986636ac20315 P a8ac3fccf4864177bba425c2fc10877d: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260924 00:56:23.948309 13072 log.cc:824] T 92912b67deb84581841986636ac20315 P 5c78d6525d0e4f35949b848e3b2bc92c: Log is configured to *not* fsync() on all Append() calls
I20260924 00:56:23.950749 13073 tablet_bootstrap.cc:492] T 92912b67deb84581841986636ac20315 P cd2c7168c6124502af2f361fabd13ef4: No bootstrap required, opened a new log
I20260924 00:56:23.951007 13073 ts_tablet_manager.cc:1397] T 92912b67deb84581841986636ac20315 P cd2c7168c6124502af2f361fabd13ef4: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260924 00:56:23.951249 13072 tablet_bootstrap.cc:492] T 92912b67deb84581841986636ac20315 P 5c78d6525d0e4f35949b848e3b2bc92c: No bootstrap required, opened a new log
I20260924 00:56:23.951489 13072 ts_tablet_manager.cc:1397] T 92912b67deb84581841986636ac20315 P 5c78d6525d0e4f35949b848e3b2bc92c: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260924 00:56:23.957034 13071 raft_consensus.cc:348] T 92912b67deb84581841986636ac20315 P a8ac3fccf4864177bba425c2fc10877d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ac3fccf4864177bba425c2fc10877d" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 44225 } } peers { permanent_uuid: "cd2c7168c6124502af2f361fabd13ef4" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 43947 } } peers { permanent_uuid: "5c78d6525d0e4f35949b848e3b2bc92c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 34959 } }
I20260924 00:56:23.957419 13071 raft_consensus.cc:374] T 92912b67deb84581841986636ac20315 P a8ac3fccf4864177bba425c2fc10877d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:23.957557 13071 raft_consensus.cc:729] T 92912b67deb84581841986636ac20315 P a8ac3fccf4864177bba425c2fc10877d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8ac3fccf4864177bba425c2fc10877d, State: Initialized, Role: FOLLOWER
I20260924 00:56:23.958200 13071 consensus_queue.cc:260] T 92912b67deb84581841986636ac20315 P a8ac3fccf4864177bba425c2fc10877d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ac3fccf4864177bba425c2fc10877d" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 44225 } } peers { permanent_uuid: "cd2c7168c6124502af2f361fabd13ef4" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 43947 } } peers { permanent_uuid: "5c78d6525d0e4f35949b848e3b2bc92c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 34959 } }
I20260924 00:56:23.959887 12921 heartbeater.cc:503] Master 127.8.64.190:43917 was elected leader, sending a full tablet report...
I20260924 00:56:23.960575 13071 ts_tablet_manager.cc:1428] T 92912b67deb84581841986636ac20315 P a8ac3fccf4864177bba425c2fc10877d: Time spent starting tablet: real 0.012s user 0.009s sys 0.004s
I20260924 00:56:23.961568 13073 raft_consensus.cc:348] T 92912b67deb84581841986636ac20315 P cd2c7168c6124502af2f361fabd13ef4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ac3fccf4864177bba425c2fc10877d" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 44225 } } peers { permanent_uuid: "cd2c7168c6124502af2f361fabd13ef4" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 43947 } } peers { permanent_uuid: "5c78d6525d0e4f35949b848e3b2bc92c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 34959 } }
I20260924 00:56:23.961688 13072 raft_consensus.cc:348] T 92912b67deb84581841986636ac20315 P 5c78d6525d0e4f35949b848e3b2bc92c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ac3fccf4864177bba425c2fc10877d" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 44225 } } peers { permanent_uuid: "cd2c7168c6124502af2f361fabd13ef4" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 43947 } } peers { permanent_uuid: "5c78d6525d0e4f35949b848e3b2bc92c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 34959 } }
I20260924 00:56:23.961989 13073 raft_consensus.cc:374] T 92912b67deb84581841986636ac20315 P cd2c7168c6124502af2f361fabd13ef4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:23.962047 13072 raft_consensus.cc:374] T 92912b67deb84581841986636ac20315 P 5c78d6525d0e4f35949b848e3b2bc92c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:23.962095 13073 raft_consensus.cc:729] T 92912b67deb84581841986636ac20315 P cd2c7168c6124502af2f361fabd13ef4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd2c7168c6124502af2f361fabd13ef4, State: Initialized, Role: FOLLOWER
I20260924 00:56:23.962138 13072 raft_consensus.cc:729] T 92912b67deb84581841986636ac20315 P 5c78d6525d0e4f35949b848e3b2bc92c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c78d6525d0e4f35949b848e3b2bc92c, State: Initialized, Role: FOLLOWER
I20260924 00:56:23.962700 13073 consensus_queue.cc:260] T 92912b67deb84581841986636ac20315 P cd2c7168c6124502af2f361fabd13ef4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ac3fccf4864177bba425c2fc10877d" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 44225 } } peers { permanent_uuid: "cd2c7168c6124502af2f361fabd13ef4" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 43947 } } peers { permanent_uuid: "5c78d6525d0e4f35949b848e3b2bc92c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 34959 } }
I20260924 00:56:23.962702 13072 consensus_queue.cc:260] T 92912b67deb84581841986636ac20315 P 5c78d6525d0e4f35949b848e3b2bc92c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ac3fccf4864177bba425c2fc10877d" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 44225 } } peers { permanent_uuid: "cd2c7168c6124502af2f361fabd13ef4" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 43947 } } peers { permanent_uuid: "5c78d6525d0e4f35949b848e3b2bc92c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 34959 } }
I20260924 00:56:23.964427 13072 ts_tablet_manager.cc:1428] T 92912b67deb84581841986636ac20315 P 5c78d6525d0e4f35949b848e3b2bc92c: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260924 00:56:23.964424 13053 heartbeater.cc:503] Master 127.8.64.190:43917 was elected leader, sending a full tablet report...
I20260924 00:56:23.964968 13073 ts_tablet_manager.cc:1428] T 92912b67deb84581841986636ac20315 P cd2c7168c6124502af2f361fabd13ef4: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
W20260924 00:56:24.085287 13054 tablet.cc:2369] T 92912b67deb84581841986636ac20315 P cd2c7168c6124502af2f361fabd13ef4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 00:56:24.104566 12922 tablet.cc:2369] T 92912b67deb84581841986636ac20315 P a8ac3fccf4864177bba425c2fc10877d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 00:56:24.108417 12789 tablet.cc:2369] T 92912b67deb84581841986636ac20315 P 5c78d6525d0e4f35949b848e3b2bc92c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:56:24.243600 13077 raft_consensus.cc:482] T 92912b67deb84581841986636ac20315 P a8ac3fccf4864177bba425c2fc10877d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:56:24.243940 13077 raft_consensus.cc:504] T 92912b67deb84581841986636ac20315 P a8ac3fccf4864177bba425c2fc10877d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ac3fccf4864177bba425c2fc10877d" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 44225 } } peers { permanent_uuid: "cd2c7168c6124502af2f361fabd13ef4" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 43947 } } peers { permanent_uuid: "5c78d6525d0e4f35949b848e3b2bc92c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 34959 } }
I20260924 00:56:24.245620 13077 leader_election.cc:288] T 92912b67deb84581841986636ac20315 P a8ac3fccf4864177bba425c2fc10877d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd2c7168c6124502af2f361fabd13ef4 (127.8.64.131:43947), 5c78d6525d0e4f35949b848e3b2bc92c (127.8.64.129:34959)
I20260924 00:56:24.253315 13070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:24.246450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:38623 (local address 127.8.64.129:34959)
0924 00:56:24.246665 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:24.246673 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:24.247130 (+ 457us) server_negotiation.cc:407] Connection header received
0924 00:56:24.247337 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:24.247344 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:24.247419 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:24.247485 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:56:24.248202 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:24.249265 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:24.251543 (+ 2278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:24.252351 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:24.252534 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:24.252610 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:24.252948 (+ 338us) server_negotiation.cc:299] Negotiation successful
0924 00:56:24.253081 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260924 00:56:24.253453 13068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:24.245942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:40215 (local address 127.8.64.131:43947)
0924 00:56:24.246169 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:24.246179 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:24.246886 (+ 707us) server_negotiation.cc:407] Connection header received
0924 00:56:24.247117 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:24.247123 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:24.247202 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:24.247298 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:56:24.247938 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:24.248746 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:24.251234 (+ 2488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:24.252483 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:24.252629 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:24.252708 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:24.253090 (+ 382us) server_negotiation.cc:299] Negotiation successful
0924 00:56:24.253221 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260924 00:56:24.253490 13081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:24.246037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:34959 (local address 127.8.64.130:38623)
0924 00:56:24.246999 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:24.247034 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:24.247155 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:24.247629 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:24.247632 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:24.247654 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:56:24.248046 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:24.248058 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:24.249411 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:24.249416 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:24.251399 (+ 1983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:24.251408 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:24.252559 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:24.252565 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:24.252777 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:24.252809 (+ 32us) client_negotiation.cc:769] Sending connection context
0924 00:56:24.252904 (+ 95us) client_negotiation.cc:240] Negotiation successful
0924 00:56:24.253040 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":832,"mutex_wait_us":7,"thread_start_us":618,"threads_started":1}
I20260924 00:56:24.254093 13080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:24.245941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.131:43947 (local address 127.8.64.130:40215)
0924 00:56:24.246730 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:24.246764 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:24.246949 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:24.247331 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:24.247337 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:24.247363 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:56:24.247755 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:24.247769 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:24.248889 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:24.248896 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:24.251064 (+ 2168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:24.251078 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:24.252768 (+ 1690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:24.252774 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:24.252924 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:24.252950 (+ 26us) client_negotiation.cc:769] Sending connection context
0924 00:56:24.253638 (+ 688us) client_negotiation.cc:240] Negotiation successful
0924 00:56:24.253793 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"mutex_wait_us":23,"thread_start_us":339,"threads_started":1}
I20260924 00:56:24.254733 12743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92912b67deb84581841986636ac20315" candidate_uuid: "a8ac3fccf4864177bba425c2fc10877d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c78d6525d0e4f35949b848e3b2bc92c" is_pre_election: true
I20260924 00:56:24.255052 13008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92912b67deb84581841986636ac20315" candidate_uuid: "a8ac3fccf4864177bba425c2fc10877d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd2c7168c6124502af2f361fabd13ef4" is_pre_election: true
I20260924 00:56:24.255251 12743 raft_consensus.cc:2454] T 92912b67deb84581841986636ac20315 P 5c78d6525d0e4f35949b848e3b2bc92c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8ac3fccf4864177bba425c2fc10877d in term 0.
I20260924 00:56:24.255553 13008 raft_consensus.cc:2454] T 92912b67deb84581841986636ac20315 P cd2c7168c6124502af2f361fabd13ef4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8ac3fccf4864177bba425c2fc10877d in term 0.
I20260924 00:56:24.255956 12811 leader_election.cc:302] T 92912b67deb84581841986636ac20315 P a8ac3fccf4864177bba425c2fc10877d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c78d6525d0e4f35949b848e3b2bc92c, a8ac3fccf4864177bba425c2fc10877d; no voters:
I20260924 00:56:24.256367 13077 raft_consensus.cc:2781] T 92912b67deb84581841986636ac20315 P a8ac3fccf4864177bba425c2fc10877d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:56:24.256513 13077 raft_consensus.cc:482] T 92912b67deb84581841986636ac20315 P a8ac3fccf4864177bba425c2fc10877d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:56:24.256637 13077 raft_consensus.cc:3037] T 92912b67deb84581841986636ac20315 P a8ac3fccf4864177bba425c2fc10877d [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:24.258849 13077 raft_consensus.cc:504] T 92912b67deb84581841986636ac20315 P a8ac3fccf4864177bba425c2fc10877d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ac3fccf4864177bba425c2fc10877d" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 44225 } } peers { permanent_uuid: "cd2c7168c6124502af2f361fabd13ef4" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 43947 } } peers { permanent_uuid: "5c78d6525d0e4f35949b848e3b2bc92c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 34959 } }
I20260924 00:56:24.259492 13077 leader_election.cc:288] T 92912b67deb84581841986636ac20315 P a8ac3fccf4864177bba425c2fc10877d [CANDIDATE]: Term 1 election: Requested vote from peers cd2c7168c6124502af2f361fabd13ef4 (127.8.64.131:43947), 5c78d6525d0e4f35949b848e3b2bc92c (127.8.64.129:34959)
I20260924 00:56:24.259824 13008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92912b67deb84581841986636ac20315" candidate_uuid: "a8ac3fccf4864177bba425c2fc10877d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd2c7168c6124502af2f361fabd13ef4"
I20260924 00:56:24.259968 12743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92912b67deb84581841986636ac20315" candidate_uuid: "a8ac3fccf4864177bba425c2fc10877d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c78d6525d0e4f35949b848e3b2bc92c"
I20260924 00:56:24.260027 13008 raft_consensus.cc:3037] T 92912b67deb84581841986636ac20315 P cd2c7168c6124502af2f361fabd13ef4 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:24.260155 12743 raft_consensus.cc:3037] T 92912b67deb84581841986636ac20315 P 5c78d6525d0e4f35949b848e3b2bc92c [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:24.262133 13008 raft_consensus.cc:2454] T 92912b67deb84581841986636ac20315 P cd2c7168c6124502af2f361fabd13ef4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8ac3fccf4864177bba425c2fc10877d in term 1.
I20260924 00:56:24.262537 12809 leader_election.cc:302] T 92912b67deb84581841986636ac20315 P a8ac3fccf4864177bba425c2fc10877d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8ac3fccf4864177bba425c2fc10877d, cd2c7168c6124502af2f361fabd13ef4; no voters:
I20260924 00:56:24.262812 13077 raft_consensus.cc:2781] T 92912b67deb84581841986636ac20315 P a8ac3fccf4864177bba425c2fc10877d [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:56:24.263223 12743 raft_consensus.cc:2454] T 92912b67deb84581841986636ac20315 P 5c78d6525d0e4f35949b848e3b2bc92c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8ac3fccf4864177bba425c2fc10877d in term 1.
I20260924 00:56:24.263626 13077 raft_consensus.cc:686] T 92912b67deb84581841986636ac20315 P a8ac3fccf4864177bba425c2fc10877d [term 1 LEADER]: Becoming Leader. State: Replica: a8ac3fccf4864177bba425c2fc10877d, State: Running, Role: LEADER
I20260924 00:56:24.264484 13077 consensus_queue.cc:237] T 92912b67deb84581841986636ac20315 P a8ac3fccf4864177bba425c2fc10877d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ac3fccf4864177bba425c2fc10877d" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 44225 } } peers { permanent_uuid: "cd2c7168c6124502af2f361fabd13ef4" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 43947 } } peers { permanent_uuid: "5c78d6525d0e4f35949b848e3b2bc92c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 34959 } }
I20260924 00:56:24.269059 12599 catalog_manager.cc:5495] T 92912b67deb84581841986636ac20315 P a8ac3fccf4864177bba425c2fc10877d reported cstate change: term changed from 0 to 1, leader changed from <none> to a8ac3fccf4864177bba425c2fc10877d (127.8.64.130). New cstate: current_term: 1 leader_uuid: "a8ac3fccf4864177bba425c2fc10877d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ac3fccf4864177bba425c2fc10877d" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 44225 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd2c7168c6124502af2f361fabd13ef4" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 43947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c78d6525d0e4f35949b848e3b2bc92c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 34959 } health_report { overall_health: UNKNOWN } } }
I20260924 00:56:24.369285 13067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:24.363970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52038 (local address 127.8.64.130:44225)
0924 00:56:24.364164 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:24.364172 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:24.364284 (+ 112us) server_negotiation.cc:407] Connection header received
0924 00:56:24.364476 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:24.364480 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:24.364554 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:24.364666 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:56:24.365123 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:24.365810 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:24.367282 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:24.367718 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:24.367808 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:24.368247 (+ 439us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:56:24.368692 (+ 445us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:56:24.368793 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:24.368941 (+ 148us) server_negotiation.cc:299] Negotiation successful
0924 00:56:24.368998 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260924 00:56:24.377278 13008 raft_consensus.cc:1261] T 92912b67deb84581841986636ac20315 P cd2c7168c6124502af2f361fabd13ef4 [term 1 FOLLOWER]: Refusing update from remote peer a8ac3fccf4864177bba425c2fc10877d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:56:24.377278 12743 raft_consensus.cc:1261] T 92912b67deb84581841986636ac20315 P 5c78d6525d0e4f35949b848e3b2bc92c [term 1 FOLLOWER]: Refusing update from remote peer a8ac3fccf4864177bba425c2fc10877d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:56:24.378274 13077 consensus_queue.cc:1035] T 92912b67deb84581841986636ac20315 P a8ac3fccf4864177bba425c2fc10877d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd2c7168c6124502af2f361fabd13ef4" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 43947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:56:24.378618 13082 consensus_queue.cc:1035] T 92912b67deb84581841986636ac20315 P a8ac3fccf4864177bba425c2fc10877d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c78d6525d0e4f35949b848e3b2bc92c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 34959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:56:24.393940 13087 mvcc.cc:205] Tried to move back new op lower bound from 7332705830399078400 to 7332705829951008768. Current Snapshot: MvccSnapshot[applied={T|T < 7332705830399078400}]
I20260924 00:56:24.404347 12600 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:56:24.404668 12600 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:56:24.408134 12600 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6517639d582a4c82aa9ea469e549beee: Sending DeleteTablet for 3 replicas of tablet 92912b67deb84581841986636ac20315
I20260924 00:56:24.409253 12856 tablet_service.cc:1514] Processing DeleteTablet for tablet 92912b67deb84581841986636ac20315 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:56:24 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:39932
I20260924 00:56:24.409246 12988 tablet_service.cc:1514] Processing DeleteTablet for tablet 92912b67deb84581841986636ac20315 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:56:24 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:58884
I20260924 00:56:24.409936 12723 tablet_service.cc:1514] Processing DeleteTablet for tablet 92912b67deb84581841986636ac20315 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:56:24 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:47760
I20260924 00:56:24.410342 13096 tablet_replica.cc:330] T 92912b67deb84581841986636ac20315 P cd2c7168c6124502af2f361fabd13ef4: stopping tablet replica
I20260924 00:56:24.410779 13096 raft_consensus.cc:2229] T 92912b67deb84581841986636ac20315 P cd2c7168c6124502af2f361fabd13ef4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:56:24.411039 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 12661
I20260924 00:56:24.411353 13096 raft_consensus.cc:2258] T 92912b67deb84581841986636ac20315 P cd2c7168c6124502af2f361fabd13ef4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:56:24.414575 13096 ts_tablet_manager.cc:1905] T 92912b67deb84581841986636ac20315 P cd2c7168c6124502af2f361fabd13ef4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:56:24.420400 13097 tablet_replica.cc:330] T 92912b67deb84581841986636ac20315 P 5c78d6525d0e4f35949b848e3b2bc92c: stopping tablet replica
I20260924 00:56:24.420838 13097 raft_consensus.cc:2229] T 92912b67deb84581841986636ac20315 P 5c78d6525d0e4f35949b848e3b2bc92c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:56:24.421294 13095 tablet_replica.cc:330] T 92912b67deb84581841986636ac20315 P a8ac3fccf4864177bba425c2fc10877d: stopping tablet replica
I20260924 00:56:24.421348 13097 raft_consensus.cc:2258] T 92912b67deb84581841986636ac20315 P 5c78d6525d0e4f35949b848e3b2bc92c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:56:24.421666 13095 raft_consensus.cc:2229] T 92912b67deb84581841986636ac20315 P a8ac3fccf4864177bba425c2fc10877d [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:56:24.422422 13095 raft_consensus.cc:2258] T 92912b67deb84581841986636ac20315 P a8ac3fccf4864177bba425c2fc10877d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:56:24.423727 13096 ts_tablet_manager.cc:1918] T 92912b67deb84581841986636ac20315 P cd2c7168c6124502af2f361fabd13ef4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:56:24.423890 13096 log.cc:1196] T 92912b67deb84581841986636ac20315 P cd2c7168c6124502af2f361fabd13ef4: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-2/wal/wals/92912b67deb84581841986636ac20315
I20260924 00:56:24.424431 13096 ts_tablet_manager.cc:1939] T 92912b67deb84581841986636ac20315 P cd2c7168c6124502af2f361fabd13ef4: Deleting consensus metadata
I20260924 00:56:24.424564 13095 ts_tablet_manager.cc:1905] T 92912b67deb84581841986636ac20315 P a8ac3fccf4864177bba425c2fc10877d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:56:24.426138 12584 catalog_manager.cc:4841] TS cd2c7168c6124502af2f361fabd13ef4 (127.8.64.131:43947): tablet 92912b67deb84581841986636ac20315 (table test-table [id=10332cedd53745c38c655fe821b0bd6d]) successfully deleted
I20260924 00:56:24.428589 13097 ts_tablet_manager.cc:1905] T 92912b67deb84581841986636ac20315 P 5c78d6525d0e4f35949b848e3b2bc92c: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 24 00:56:24 dist-test-slave-mmhs krb5kdc[12556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.129@KRBTEST.COM
I20260924 00:56:24.432025 13095 ts_tablet_manager.cc:1918] T 92912b67deb84581841986636ac20315 P a8ac3fccf4864177bba425c2fc10877d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:56:24.432184 13095 log.cc:1196] T 92912b67deb84581841986636ac20315 P a8ac3fccf4864177bba425c2fc10877d: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-1/wal/wals/92912b67deb84581841986636ac20315
I20260924 00:56:24.432788 13095 ts_tablet_manager.cc:1939] T 92912b67deb84581841986636ac20315 P a8ac3fccf4864177bba425c2fc10877d: Deleting consensus metadata
I20260924 00:56:24.434490 12586 catalog_manager.cc:4841] TS a8ac3fccf4864177bba425c2fc10877d (127.8.64.130:44225): tablet 92912b67deb84581841986636ac20315 (table test-table [id=10332cedd53745c38c655fe821b0bd6d]) successfully deleted
I20260924 00:56:24.436277 13097 ts_tablet_manager.cc:1918] T 92912b67deb84581841986636ac20315 P 5c78d6525d0e4f35949b848e3b2bc92c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:56:24.436450 13097 log.cc:1196] T 92912b67deb84581841986636ac20315 P 5c78d6525d0e4f35949b848e3b2bc92c: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790211343099518-8450-0/minicluster-data/ts-0/wal/wals/92912b67deb84581841986636ac20315
I20260924 00:56:24.437036 13097 ts_tablet_manager.cc:1939] T 92912b67deb84581841986636ac20315 P 5c78d6525d0e4f35949b848e3b2bc92c: Deleting consensus metadata
I20260924 00:56:24.438719 12586 catalog_manager.cc:4841] TS 5c78d6525d0e4f35949b848e3b2bc92c (127.8.64.129:34959): tablet 92912b67deb84581841986636ac20315 (table test-table [id=10332cedd53745c38c655fe821b0bd6d]) successfully deleted
I20260924 00:56:24.440294 13070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:24.412385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34642 (local address 127.8.64.129:34959)
0924 00:56:24.412605 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:24.412614 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:24.420645 (+ 8031us) server_negotiation.cc:407] Connection header received
0924 00:56:24.420884 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:24.420892 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:24.420986 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:24.421099 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:24.423233 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:24.424365 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:24.426336 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:24.426969 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:24.427078 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:24.432458 (+ 5380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:24.432475 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:24.432494 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:24.432537 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:24.435906 (+ 3369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:24.436809 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:24.436818 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:24.436822 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:24.437094 (+ 272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:24.437792 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:24.437798 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:24.437801 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:24.438543 (+ 742us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:24.438956 (+ 413us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:24.439726 (+ 770us) server_negotiation.cc:299] Negotiation successful
0924 00:56:24.439932 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260924 00:56:24.440915 12783 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:56:24.559535 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 12661
I20260924 00:56:24.577266 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 12794
Sep 24 00:56:24 dist-test-slave-mmhs krb5kdc[12556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.130@KRBTEST.COM
I20260924 00:56:24.594553 13067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:24.578046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52052 (local address 127.8.64.130:44225)
0924 00:56:24.578217 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:24.578221 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:24.578352 (+ 131us) server_negotiation.cc:407] Connection header received
0924 00:56:24.578525 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:24.578529 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:24.578599 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:24.578681 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:24.580237 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:24.581113 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:24.582924 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:24.583329 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:24.583425 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:24.588074 (+ 4649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:24.588088 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:24.588103 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:24.588132 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:24.591623 (+ 3491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:24.592279 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:24.592286 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:24.592288 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:24.592423 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:24.592796 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:24.592807 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:24.592810 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:24.593202 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:24.593571 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:24.593805 (+ 234us) server_negotiation.cc:299] Negotiation successful
0924 00:56:24.594023 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260924 00:56:24.595455 12916 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:56:24.706831 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 12794
I20260924 00:56:24.726111 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 12926
Sep 24 00:56:24 dist-test-slave-mmhs krb5kdc[12556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.131@KRBTEST.COM
I20260924 00:56:24.740619 13068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:24.726877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33946 (local address 127.8.64.131:43947)
0924 00:56:24.727040 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:24.727045 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:24.727107 (+ 62us) server_negotiation.cc:407] Connection header received
0924 00:56:24.727299 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:24.727304 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:24.727372 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:24.727471 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:24.728643 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:24.729342 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:24.730552 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:24.730937 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:24.731030 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:24.735119 (+ 4089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:24.735131 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:24.735145 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:24.735173 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:24.737954 (+ 2781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:24.738872 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:24.738881 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:24.738883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:24.739042 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:24.739379 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:24.739383 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:24.739385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:24.739743 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:24.739958 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:24.740199 (+ 241us) server_negotiation.cc:299] Negotiation successful
0924 00:56:24.740350 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260924 00:56:24.741272 13048 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:56:24.828526 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 12926
I20260924 00:56:24.845028 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 12569
I20260924 00:56:24.846153 12630 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:56:24.951522 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 12569
2026-09-24T00:56:24Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4438 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-24T00:56:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:56:25Z Disabled control of system clock
I20260924 00:56:25.025907 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:34099
--webserver_interface=127.8.64.190
--webserver_port=0
--builtin_ntp_servers=127.8.64.148:36131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:34099
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260924 00:56:25.345288 13106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:25.345554 13106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:25.345635 13106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:25.353845 13106 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:56:25.353998 13106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:25.354082 13106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:25.354167 13106 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:56:25.354235 13106 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:56:25.364836 13106 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:36131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:34099
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:34099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:25.366110 13106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:25.367920 13106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:25.379570 13111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:25.379556 13112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:25.379585 13115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:25.380816 13106 server_base.cc:1034] running on GCE node
I20260924 00:56:25.381603 13106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:25.383071 13106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:25.384291 13106 hybrid_clock.cc:648] HybridClock initialized: now 1790211385384255 us; error 35 us; skew 500 ppm
I20260924 00:56:25.384653 13106 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 00:56:25.387072 13106 webserver.cc:460] Webserver started at http://127.8.64.190:44877/ using document root <none> and password file <none>
I20260924 00:56:25.387741 13106 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:25.387827 13106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:25.388139 13106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:56:25.390970 13106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "b42291201c374266930125689ef020a9"
format_stamp: "Formatted at 2026-09-24 00:56:25 on dist-test-slave-mmhs"
server_key: "25c83d90cabe163dfb1acf579161288e"
server_key_iv: "bdd20766429382e1e18fd1cf87e9ac53"
server_key_version: "encryptionkey@0"
I20260924 00:56:25.391741 13106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "b42291201c374266930125689ef020a9"
format_stamp: "Formatted at 2026-09-24 00:56:25 on dist-test-slave-mmhs"
server_key: "25c83d90cabe163dfb1acf579161288e"
server_key_iv: "bdd20766429382e1e18fd1cf87e9ac53"
server_key_version: "encryptionkey@0"
I20260924 00:56:25.396834 13106 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260924 00:56:25.400274 13121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:25.401615 13106 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260924 00:56:25.401785 13106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "b42291201c374266930125689ef020a9"
format_stamp: "Formatted at 2026-09-24 00:56:25 on dist-test-slave-mmhs"
server_key: "25c83d90cabe163dfb1acf579161288e"
server_key_iv: "bdd20766429382e1e18fd1cf87e9ac53"
server_key_version: "encryptionkey@0"
I20260924 00:56:25.401976 13106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:25.407641 13106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:25.408399 13106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:25.408704 13106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:25.426776 13106 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.190:34099
I20260924 00:56:25.426823 13172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.190:34099 every 8 connection(s)
I20260924 00:56:25.428052 13106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 00:56:25.431669 13173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:25.436770 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 13106
I20260924 00:56:25.436949 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 00:56:25.437711 8450 external_mini_cluster.cc:1428] Setting key 0fe217bae0943c17d130e57dbb4b02a4
I20260924 00:56:25.444063 13173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b42291201c374266930125689ef020a9: Bootstrap starting.
I20260924 00:56:25.447242 13173 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b42291201c374266930125689ef020a9: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:25.447980 13176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:25.439859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54146 (local address 127.8.64.190:34099)
0924 00:56:25.441376 (+ 1517us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:25.441392 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:25.441457 (+ 65us) server_negotiation.cc:407] Connection header received
0924 00:56:25.443677 (+ 2220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:25.443714 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:25.444225 (+ 511us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:25.444661 (+ 436us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:25.446378 (+ 1717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:25.446389 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:25.446407 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:56:25.446422 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:25.446482 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:56:25.446498 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:25.446688 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:25.446819 (+ 131us) server_negotiation.cc:299] Negotiation successful
0924 00:56:25.447069 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":908,"thread_start_us":513,"threads_started":1}
I20260924 00:56:25.448351 13173 log.cc:824] T 00000000000000000000000000000000 P b42291201c374266930125689ef020a9: Log is configured to *not* fsync() on all Append() calls
I20260924 00:56:25.451195 13173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b42291201c374266930125689ef020a9: No bootstrap required, opened a new log
I20260924 00:56:25.458904 13173 raft_consensus.cc:348] T 00000000000000000000000000000000 P b42291201c374266930125689ef020a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b42291201c374266930125689ef020a9" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 34099 } }
I20260924 00:56:25.459300 13173 raft_consensus.cc:374] T 00000000000000000000000000000000 P b42291201c374266930125689ef020a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:25.459407 13173 raft_consensus.cc:729] T 00000000000000000000000000000000 P b42291201c374266930125689ef020a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b42291201c374266930125689ef020a9, State: Initialized, Role: FOLLOWER
I20260924 00:56:25.459929 13173 consensus_queue.cc:260] T 00000000000000000000000000000000 P b42291201c374266930125689ef020a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b42291201c374266930125689ef020a9" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 34099 } }
I20260924 00:56:25.460168 13173 raft_consensus.cc:388] T 00000000000000000000000000000000 P b42291201c374266930125689ef020a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:56:25.460327 13173 raft_consensus.cc:482] T 00000000000000000000000000000000 P b42291201c374266930125689ef020a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:56:25.460495 13173 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b42291201c374266930125689ef020a9 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:25.462658 13173 raft_consensus.cc:504] T 00000000000000000000000000000000 P b42291201c374266930125689ef020a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b42291201c374266930125689ef020a9" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 34099 } }
I20260924 00:56:25.463173 13173 leader_election.cc:302] T 00000000000000000000000000000000 P b42291201c374266930125689ef020a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b42291201c374266930125689ef020a9; no voters:
I20260924 00:56:25.463742 13173 leader_election.cc:288] T 00000000000000000000000000000000 P b42291201c374266930125689ef020a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:56:25.463877 13178 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b42291201c374266930125689ef020a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:56:25.464989 13178 raft_consensus.cc:686] T 00000000000000000000000000000000 P b42291201c374266930125689ef020a9 [term 1 LEADER]: Becoming Leader. State: Replica: b42291201c374266930125689ef020a9, State: Running, Role: LEADER
I20260924 00:56:25.465562 13178 consensus_queue.cc:237] T 00000000000000000000000000000000 P b42291201c374266930125689ef020a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b42291201c374266930125689ef020a9" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 34099 } }
I20260924 00:56:25.465986 13173 sys_catalog.cc:564] T 00000000000000000000000000000000 P b42291201c374266930125689ef020a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:56:25.468744 13180 sys_catalog.cc:455] T 00000000000000000000000000000000 P b42291201c374266930125689ef020a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b42291201c374266930125689ef020a9. Latest consensus state: current_term: 1 leader_uuid: "b42291201c374266930125689ef020a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b42291201c374266930125689ef020a9" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 34099 } } }
I20260924 00:56:25.469103 13180 sys_catalog.cc:458] T 00000000000000000000000000000000 P b42291201c374266930125689ef020a9 [sys.catalog]: This master's current role is: LEADER
I20260924 00:56:25.468762 13179 sys_catalog.cc:455] T 00000000000000000000000000000000 P b42291201c374266930125689ef020a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b42291201c374266930125689ef020a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b42291201c374266930125689ef020a9" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 34099 } } }
I20260924 00:56:25.469214 13179 sys_catalog.cc:458] T 00000000000000000000000000000000 P b42291201c374266930125689ef020a9 [sys.catalog]: This master's current role is: LEADER
I20260924 00:56:25.470577 13185 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:56:25.475539 13185 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:56:25.485494 13185 catalog_manager.cc:1347] Generated new cluster ID: 32e327cfbe394235ae74171a309766cf
I20260924 00:56:25.485595 13185 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:56:25.515434 13185 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:56:25.515861 13185 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:56:25.528568 13185 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b42291201c374266930125689ef020a9: Generated new TSK 0
I20260924 00:56:25.529588 13185 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 00:56:25.544184 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.129:0
--local_ip_for_outbound_sockets=127.8.64.129
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:34099
--builtin_ntp_servers=127.8.64.148:36131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260924 00:56:25.879923 13197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:25.880206 13197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:25.880295 13197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:25.887380 13197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:25.887478 13197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:25.887585 13197 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.129
I20260924 00:56:25.896515 13197 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:36131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:34099
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.129
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:25.897722 13197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:25.899477 13197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:25.912246 13202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:25.912292 13205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:25.912786 13203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:25.913079 13197 server_base.cc:1034] running on GCE node
I20260924 00:56:25.913640 13197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:25.914564 13197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:25.915812 13197 hybrid_clock.cc:648] HybridClock initialized: now 1790211385915774 us; error 51 us; skew 500 ppm
I20260924 00:56:25.916121 13197 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 00:56:25.918423 13197 webserver.cc:460] Webserver started at http://127.8.64.129:45137/ using document root <none> and password file <none>
I20260924 00:56:25.919138 13197 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:25.919271 13197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:25.919600 13197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:56:25.922526 13197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-0/data/instance:
uuid: "1186876c38d64aa4b21d79b8ff6529e5"
format_stamp: "Formatted at 2026-09-24 00:56:25 on dist-test-slave-mmhs"
server_key: "2b1fac269ee71b24bef5b16ee1912dba"
server_key_iv: "aa09a76466eaba00669ae5c291739be0"
server_key_version: "encryptionkey@0"
I20260924 00:56:25.923332 13197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance:
uuid: "1186876c38d64aa4b21d79b8ff6529e5"
format_stamp: "Formatted at 2026-09-24 00:56:25 on dist-test-slave-mmhs"
server_key: "2b1fac269ee71b24bef5b16ee1912dba"
server_key_iv: "aa09a76466eaba00669ae5c291739be0"
server_key_version: "encryptionkey@0"
I20260924 00:56:25.928296 13197 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260924 00:56:25.931588 13211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:25.932806 13197 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260924 00:56:25.932977 13197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-0/wal
uuid: "1186876c38d64aa4b21d79b8ff6529e5"
format_stamp: "Formatted at 2026-09-24 00:56:25 on dist-test-slave-mmhs"
server_key: "2b1fac269ee71b24bef5b16ee1912dba"
server_key_iv: "aa09a76466eaba00669ae5c291739be0"
server_key_version: "encryptionkey@0"
I20260924 00:56:25.933207 13197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:25.938995 13197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:25.939906 13197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:25.940182 13197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:25.941075 13197 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:56:25.942492 13197 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:56:25.942624 13197 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:25.942749 13197 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:56:25.942839 13197 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:25.975888 13197 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.129:37291
I20260924 00:56:25.975937 13323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.129:37291 every 8 connection(s)
I20260924 00:56:25.977181 13197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
I20260924 00:56:25.984980 13326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:25.979593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:44443 (local address 127.8.64.190:34099)
0924 00:56:25.980022 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:25.980027 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:25.981050 (+ 1023us) server_negotiation.cc:407] Connection header received
0924 00:56:25.982774 (+ 1724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:25.982780 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:25.982846 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:25.982962 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:25.984211 (+ 1249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:25.984217 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:25.984222 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:56:25.984239 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:25.984297 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:56:25.984309 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:25.984430 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:25.984627 (+ 197us) server_negotiation.cc:299] Negotiation successful
0924 00:56:25.984692 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":242,"threads_started":1}
I20260924 00:56:25.985867 13327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:25.980041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:34099 (local address 127.8.64.129:44443)
0924 00:56:25.980909 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:25.980944 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:25.982500 (+ 1556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:25.983190 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:25.983197 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:25.983293 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:25.983931 (+ 638us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 00:56:25.983934 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:25.983969 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 00:56:25.984047 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:25.984403 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:25.984413 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:25.984421 (+ 8us) client_negotiation.cc:769] Sending connection context
0924 00:56:25.984696 (+ 275us) client_negotiation.cc:240] Negotiation successful
0924 00:56:25.984937 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":272,"threads_started":1}
I20260924 00:56:25.986743 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 13197
I20260924 00:56:25.986937 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance
I20260924 00:56:25.987396 13324 heartbeater.cc:347] Connected to a master server at 127.8.64.190:34099
I20260924 00:56:25.987727 13324 heartbeater.cc:464] Registering TS with master...
I20260924 00:56:25.987762 8450 external_mini_cluster.cc:1428] Setting key 0135860cb4cd310e94df9b44cbbb0790
I20260924 00:56:25.988411 13324 heartbeater.cc:511] Master 127.8.64.190:34099 requested a full tablet report, sending...
I20260924 00:56:25.990665 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.130:0
--local_ip_for_outbound_sockets=127.8.64.130
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:34099
--builtin_ntp_servers=127.8.64.148:36131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260924 00:56:25.990677 13138 ts_manager.cc:194] Registered new tserver with Master: 1186876c38d64aa4b21d79b8ff6529e5 (127.8.64.129:37291)
W20260924 00:56:26.312378 13328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:26.312638 13328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:26.312813 13328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:26.319996 13328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:26.320086 13328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:26.320230 13328 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.130
I20260924 00:56:26.329059 13328 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:36131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:34099
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.130
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:26.330286 13328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:26.331995 13328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:26.344782 13333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:26.344820 13334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:26.344854 13336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:26.345520 13328 server_base.cc:1034] running on GCE node
I20260924 00:56:26.346065 13328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:26.346973 13328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:26.348167 13328 hybrid_clock.cc:648] HybridClock initialized: now 1790211386348128 us; error 32 us; skew 500 ppm
I20260924 00:56:26.348487 13328 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 00:56:26.350657 13328 webserver.cc:460] Webserver started at http://127.8.64.130:41005/ using document root <none> and password file <none>
I20260924 00:56:26.351328 13328 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:26.351437 13328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:26.351758 13328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:56:26.354504 13328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-1/data/instance:
uuid: "bfb0bf9113574a099141f94538e51f32"
format_stamp: "Formatted at 2026-09-24 00:56:26 on dist-test-slave-mmhs"
server_key: "f7fb4e76da90a3b32ff08ae5f973f181"
server_key_iv: "e45f5b4d40e2203ea3bccbbfec8b9c34"
server_key_version: "encryptionkey@0"
I20260924 00:56:26.355304 13328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance:
uuid: "bfb0bf9113574a099141f94538e51f32"
format_stamp: "Formatted at 2026-09-24 00:56:26 on dist-test-slave-mmhs"
server_key: "f7fb4e76da90a3b32ff08ae5f973f181"
server_key_iv: "e45f5b4d40e2203ea3bccbbfec8b9c34"
server_key_version: "encryptionkey@0"
I20260924 00:56:26.360270 13328 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260924 00:56:26.363488 13342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:26.364732 13328 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 00:56:26.364907 13328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-1/wal
uuid: "bfb0bf9113574a099141f94538e51f32"
format_stamp: "Formatted at 2026-09-24 00:56:26 on dist-test-slave-mmhs"
server_key: "f7fb4e76da90a3b32ff08ae5f973f181"
server_key_iv: "e45f5b4d40e2203ea3bccbbfec8b9c34"
server_key_version: "encryptionkey@0"
I20260924 00:56:26.365120 13328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:26.372478 13328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:26.373483 13328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:26.373790 13328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:26.375052 13328 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:56:26.376979 13328 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:56:26.377094 13328 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:26.377221 13328 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:56:26.377310 13328 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:26.410729 13328 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.130:36961
I20260924 00:56:26.410780 13456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.130:36961 every 8 connection(s)
I20260924 00:56:26.412001 13328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
I20260924 00:56:26.420145 13326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:26.414588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:37329 (local address 127.8.64.190:34099)
0924 00:56:26.414768 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:26.414772 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:26.416257 (+ 1485us) server_negotiation.cc:407] Connection header received
0924 00:56:26.417910 (+ 1653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:26.417916 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:26.417988 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:26.418097 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:26.419427 (+ 1330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:26.419432 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:26.419438 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:56:26.419454 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:26.419493 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:56:26.419505 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:26.419630 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:26.419816 (+ 186us) server_negotiation.cc:299] Negotiation successful
0924 00:56:26.419881 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260924 00:56:26.420423 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 13328
I20260924 00:56:26.420591 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance
I20260924 00:56:26.421015 8450 external_mini_cluster.cc:1428] Setting key ddd1645cf0ba899905daa0cfd359dbab
I20260924 00:56:26.421157 13459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:26.415105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:34099 (local address 127.8.64.130:37329)
0924 00:56:26.416006 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:26.416047 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:26.417655 (+ 1608us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:26.418367 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:26.418374 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:26.418466 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:26.419138 (+ 672us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 00:56:26.419140 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:26.419176 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 00:56:26.419261 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:26.419597 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:26.419609 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:26.419617 (+ 8us) client_negotiation.cc:769] Sending connection context
0924 00:56:26.419895 (+ 278us) client_negotiation.cc:240] Negotiation successful
0924 00:56:26.420154 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":279,"threads_started":1}
I20260924 00:56:26.422578 13457 heartbeater.cc:347] Connected to a master server at 127.8.64.190:34099
I20260924 00:56:26.422850 13457 heartbeater.cc:464] Registering TS with master...
I20260924 00:56:26.423377 13457 heartbeater.cc:511] Master 127.8.64.190:34099 requested a full tablet report, sending...
I20260924 00:56:26.423985 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.131:0
--local_ip_for_outbound_sockets=127.8.64.131
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:34099
--builtin_ntp_servers=127.8.64.148:36131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260924 00:56:26.424708 13137 ts_manager.cc:194] Registered new tserver with Master: bfb0bf9113574a099141f94538e51f32 (127.8.64.130:36961)
W20260924 00:56:26.747619 13460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:26.747879 13460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:26.748027 13460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:26.755177 13460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:26.755273 13460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:26.755468 13460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.131
I20260924 00:56:26.764446 13460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:36131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:34099
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.131
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:26.765697 13460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:26.767650 13460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:26.780807 13466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:26.780980 13468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:26.781363 13465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:26.781771 13460 server_base.cc:1034] running on GCE node
I20260924 00:56:26.782423 13460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:26.783267 13460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:26.784463 13460 hybrid_clock.cc:648] HybridClock initialized: now 1790211386784409 us; error 54 us; skew 500 ppm
I20260924 00:56:26.784719 13460 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 00:56:26.786969 13460 webserver.cc:460] Webserver started at http://127.8.64.131:37345/ using document root <none> and password file <none>
I20260924 00:56:26.787565 13460 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:26.787648 13460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:26.787907 13460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:56:26.790648 13460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-2/data/instance:
uuid: "6ca7821dd7c144ddbb92ad9311b50970"
format_stamp: "Formatted at 2026-09-24 00:56:26 on dist-test-slave-mmhs"
server_key: "065f3e5be10af24187a0f8eaa4220858"
server_key_iv: "ed52c230d69ef0d1d77447896c1e2dca"
server_key_version: "encryptionkey@0"
I20260924 00:56:26.791437 13460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance:
uuid: "6ca7821dd7c144ddbb92ad9311b50970"
format_stamp: "Formatted at 2026-09-24 00:56:26 on dist-test-slave-mmhs"
server_key: "065f3e5be10af24187a0f8eaa4220858"
server_key_iv: "ed52c230d69ef0d1d77447896c1e2dca"
server_key_version: "encryptionkey@0"
I20260924 00:56:26.796682 13460 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260924 00:56:26.799966 13474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:26.801246 13460 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260924 00:56:26.801366 13460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-2/wal
uuid: "6ca7821dd7c144ddbb92ad9311b50970"
format_stamp: "Formatted at 2026-09-24 00:56:26 on dist-test-slave-mmhs"
server_key: "065f3e5be10af24187a0f8eaa4220858"
server_key_iv: "ed52c230d69ef0d1d77447896c1e2dca"
server_key_version: "encryptionkey@0"
I20260924 00:56:26.801548 13460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:26.806968 13460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:26.807665 13460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:26.807888 13460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:26.808682 13460 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:56:26.809945 13460 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:56:26.810029 13460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:26.810104 13460 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:56:26.810158 13460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:26.841989 13460 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.131:37537
I20260924 00:56:26.842031 13587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.131:37537 every 8 connection(s)
I20260924 00:56:26.843267 13460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
I20260924 00:56:26.844136 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 13460
I20260924 00:56:26.844303 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance
I20260924 00:56:26.844856 8450 external_mini_cluster.cc:1428] Setting key 2c751471cb20d86bad8ad2c08e082272
I20260924 00:56:26.851905 13326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:26.846093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:52711 (local address 127.8.64.190:34099)
0924 00:56:26.846328 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:26.846335 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:26.847963 (+ 1628us) server_negotiation.cc:407] Connection header received
0924 00:56:26.849671 (+ 1708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:26.849676 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:26.849741 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:26.849834 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:26.851177 (+ 1343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:26.851183 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:26.851187 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:56:26.851206 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:26.851248 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:56:26.851267 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:26.851393 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:26.851583 (+ 190us) server_negotiation.cc:299] Negotiation successful
0924 00:56:26.851648 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260924 00:56:26.852797 13590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:26.846681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:34099 (local address 127.8.64.131:52711)
0924 00:56:26.847802 (+ 1121us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:26.847864 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:26.849420 (+ 1556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:26.850069 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:26.850077 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:26.850168 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:26.850894 (+ 726us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 00:56:26.850896 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:26.850931 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 00:56:26.851014 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:26.851362 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:26.851374 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:26.851382 (+ 8us) client_negotiation.cc:769] Sending connection context
0924 00:56:26.851664 (+ 282us) client_negotiation.cc:240] Negotiation successful
0924 00:56:26.851923 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":361,"threads_started":1}
I20260924 00:56:26.854049 13588 heartbeater.cc:347] Connected to a master server at 127.8.64.190:34099
I20260924 00:56:26.854391 13588 heartbeater.cc:464] Registering TS with master...
I20260924 00:56:26.854959 13588 heartbeater.cc:511] Master 127.8.64.190:34099 requested a full tablet report, sending...
I20260924 00:56:26.856303 13137 ts_manager.cc:194] Registered new tserver with Master: 6ca7821dd7c144ddbb92ad9311b50970 (127.8.64.131:37537)
I20260924 00:56:26.860569 8450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 00:56:26.867820 13326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:26.865053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54148 (local address 127.8.64.190:34099)
0924 00:56:26.865252 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:26.865261 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:26.865686 (+ 425us) server_negotiation.cc:407] Connection header received
0924 00:56:26.865900 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:26.865912 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:26.865964 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:26.866067 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:26.867137 (+ 1070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:26.867141 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:26.867145 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:56:26.867160 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:26.867189 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:56:26.867201 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:26.867306 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:26.867459 (+ 153us) server_negotiation.cc:299] Negotiation successful
0924 00:56:26.867522 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260924 00:56:26.871946 13137 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54148:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:56:26.878883 13137 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:56:26.909907 13605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:26.891596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56714 (local address 127.8.64.129:37291)
0924 00:56:26.904712 (+ 13116us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:26.904725 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:26.906269 (+ 1544us) server_negotiation.cc:407] Connection header received
0924 00:56:26.906382 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:26.906390 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:26.906634 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:26.906792 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:26.908902 (+ 2110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:26.908912 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:26.908926 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:56:26.908946 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:26.909008 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:56:26.909028 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:26.909161 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:26.909400 (+ 239us) server_negotiation.cc:299] Negotiation successful
0924 00:56:26.909509 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12968,"thread_start_us":12851,"threads_started":1}
I20260924 00:56:26.910010 13600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:26.891331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:37291 (local address 127.0.0.1:56714)
0924 00:56:26.906042 (+ 14711us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:26.906060 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:26.906154 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:26.907715 (+ 1561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:26.907720 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:26.907747 (+ 27us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:26.908653 (+ 906us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 00:56:26.908656 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:26.908672 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 00:56:26.908729 (+ 57us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:26.909137 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:26.909147 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:26.909156 (+ 9us) client_negotiation.cc:769] Sending connection context
0924 00:56:26.909345 (+ 189us) client_negotiation.cc:240] Negotiation successful
0924 00:56:26.909441 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14524,"thread_start_us":12926,"threads_started":1}
I20260924 00:56:26.910225 13603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:26.890687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.131:37537 (local address 127.0.0.1:59338)
0924 00:56:26.903917 (+ 13230us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:26.903969 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:26.904118 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:26.907642 (+ 3524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:26.907647 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:26.907671 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:26.908855 (+ 1184us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 00:56:26.908858 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:26.908871 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 00:56:26.908906 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:26.909423 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:26.909427 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:26.909432 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 00:56:26.909560 (+ 128us) client_negotiation.cc:240] Negotiation successful
0924 00:56:26.909611 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12884,"thread_start_us":15052,"threads_started":1}
I20260924 00:56:26.910604 13602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:26.891111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.130:36961 (local address 127.0.0.1:36336)
0924 00:56:26.905377 (+ 14266us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:26.905401 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:26.905505 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:26.906838 (+ 1333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:26.906846 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:26.906939 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:26.908497 (+ 1558us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 00:56:26.908503 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:26.908537 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 00:56:26.908634 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:26.909571 (+ 937us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:26.909577 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:26.909583 (+ 6us) client_negotiation.cc:769] Sending connection context
0924 00:56:26.909762 (+ 179us) client_negotiation.cc:240] Negotiation successful
0924 00:56:26.909821 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14088,"thread_start_us":13963,"threads_started":1}
I20260924 00:56:26.910640 13604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:26.891471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36336 (local address 127.8.64.130:36961)
0924 00:56:26.906387 (+ 14916us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:26.906395 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:26.906418 (+ 23us) server_negotiation.cc:407] Connection header received
0924 00:56:26.906476 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:26.906483 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:26.906634 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:26.906792 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:26.909059 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:26.909069 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:26.909084 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:56:26.909104 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:26.909153 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:56:26.909175 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:26.909479 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:26.910116 (+ 637us) server_negotiation.cc:299] Negotiation successful
0924 00:56:26.910207 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14673,"thread_start_us":14571,"threads_started":1}
I20260924 00:56:26.911775 13601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:26.890957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59338 (local address 127.8.64.131:37537)
0924 00:56:26.906791 (+ 15834us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:26.906798 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:26.906831 (+ 33us) server_negotiation.cc:407] Connection header received
0924 00:56:26.906890 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:26.906896 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:26.907101 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:26.907653 (+ 552us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:26.909192 (+ 1539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:26.909199 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:26.909211 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:56:26.909230 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:26.909285 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:56:26.909305 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:26.911176 (+ 1871us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:26.911242 (+ 66us) server_negotiation.cc:299] Negotiation successful
0924 00:56:26.911330 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15652,"thread_start_us":15503,"threads_started":1}
I20260924 00:56:26.914425 13392 tablet_service.cc:1467] Processing CreateTablet for tablet 90c8f0bdb53e475f95c176dad316c220 (DEFAULT_TABLE table=test-table [id=5f896c1213b64d0a897fb7e37b1acd71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:56:26.914424 13259 tablet_service.cc:1467] Processing CreateTablet for tablet 90c8f0bdb53e475f95c176dad316c220 (DEFAULT_TABLE table=test-table [id=5f896c1213b64d0a897fb7e37b1acd71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:56:26.915191 13522 tablet_service.cc:1467] Processing CreateTablet for tablet 90c8f0bdb53e475f95c176dad316c220 (DEFAULT_TABLE table=test-table [id=5f896c1213b64d0a897fb7e37b1acd71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:56:26.915864 13392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90c8f0bdb53e475f95c176dad316c220. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:26.915864 13259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90c8f0bdb53e475f95c176dad316c220. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:26.916400 13522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90c8f0bdb53e475f95c176dad316c220. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:26.927469 13606 tablet_bootstrap.cc:492] T 90c8f0bdb53e475f95c176dad316c220 P 6ca7821dd7c144ddbb92ad9311b50970: Bootstrap starting.
I20260924 00:56:26.931137 13606 tablet_bootstrap.cc:654] T 90c8f0bdb53e475f95c176dad316c220 P 6ca7821dd7c144ddbb92ad9311b50970: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:26.931595 13607 tablet_bootstrap.cc:492] T 90c8f0bdb53e475f95c176dad316c220 P 1186876c38d64aa4b21d79b8ff6529e5: Bootstrap starting.
I20260924 00:56:26.932621 13606 log.cc:824] T 90c8f0bdb53e475f95c176dad316c220 P 6ca7821dd7c144ddbb92ad9311b50970: Log is configured to *not* fsync() on all Append() calls
I20260924 00:56:26.933320 13608 tablet_bootstrap.cc:492] T 90c8f0bdb53e475f95c176dad316c220 P bfb0bf9113574a099141f94538e51f32: Bootstrap starting.
I20260924 00:56:26.935878 13607 tablet_bootstrap.cc:654] T 90c8f0bdb53e475f95c176dad316c220 P 1186876c38d64aa4b21d79b8ff6529e5: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:26.936224 13608 tablet_bootstrap.cc:654] T 90c8f0bdb53e475f95c176dad316c220 P bfb0bf9113574a099141f94538e51f32: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:26.937350 13608 log.cc:824] T 90c8f0bdb53e475f95c176dad316c220 P bfb0bf9113574a099141f94538e51f32: Log is configured to *not* fsync() on all Append() calls
I20260924 00:56:26.937489 13607 log.cc:824] T 90c8f0bdb53e475f95c176dad316c220 P 1186876c38d64aa4b21d79b8ff6529e5: Log is configured to *not* fsync() on all Append() calls
I20260924 00:56:26.940775 13608 tablet_bootstrap.cc:492] T 90c8f0bdb53e475f95c176dad316c220 P bfb0bf9113574a099141f94538e51f32: No bootstrap required, opened a new log
I20260924 00:56:26.940865 13606 tablet_bootstrap.cc:492] T 90c8f0bdb53e475f95c176dad316c220 P 6ca7821dd7c144ddbb92ad9311b50970: No bootstrap required, opened a new log
I20260924 00:56:26.940866 13607 tablet_bootstrap.cc:492] T 90c8f0bdb53e475f95c176dad316c220 P 1186876c38d64aa4b21d79b8ff6529e5: No bootstrap required, opened a new log
I20260924 00:56:26.941045 13608 ts_tablet_manager.cc:1397] T 90c8f0bdb53e475f95c176dad316c220 P bfb0bf9113574a099141f94538e51f32: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260924 00:56:26.941133 13606 ts_tablet_manager.cc:1397] T 90c8f0bdb53e475f95c176dad316c220 P 6ca7821dd7c144ddbb92ad9311b50970: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.005s
I20260924 00:56:26.941120 13607 ts_tablet_manager.cc:1397] T 90c8f0bdb53e475f95c176dad316c220 P 1186876c38d64aa4b21d79b8ff6529e5: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260924 00:56:26.948447 13608 raft_consensus.cc:348] T 90c8f0bdb53e475f95c176dad316c220 P bfb0bf9113574a099141f94538e51f32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ca7821dd7c144ddbb92ad9311b50970" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 37537 } } peers { permanent_uuid: "bfb0bf9113574a099141f94538e51f32" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 36961 } } peers { permanent_uuid: "1186876c38d64aa4b21d79b8ff6529e5" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37291 } }
I20260924 00:56:26.948849 13608 raft_consensus.cc:374] T 90c8f0bdb53e475f95c176dad316c220 P bfb0bf9113574a099141f94538e51f32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:26.948962 13608 raft_consensus.cc:729] T 90c8f0bdb53e475f95c176dad316c220 P bfb0bf9113574a099141f94538e51f32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfb0bf9113574a099141f94538e51f32, State: Initialized, Role: FOLLOWER
I20260924 00:56:26.949630 13608 consensus_queue.cc:260] T 90c8f0bdb53e475f95c176dad316c220 P bfb0bf9113574a099141f94538e51f32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ca7821dd7c144ddbb92ad9311b50970" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 37537 } } peers { permanent_uuid: "bfb0bf9113574a099141f94538e51f32" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 36961 } } peers { permanent_uuid: "1186876c38d64aa4b21d79b8ff6529e5" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37291 } }
I20260924 00:56:26.950765 13457 heartbeater.cc:503] Master 127.8.64.190:34099 was elected leader, sending a full tablet report...
I20260924 00:56:26.951460 13608 ts_tablet_manager.cc:1428] T 90c8f0bdb53e475f95c176dad316c220 P bfb0bf9113574a099141f94538e51f32: Time spent starting tablet: real 0.010s user 0.011s sys 0.000s
I20260924 00:56:26.952776 13607 raft_consensus.cc:348] T 90c8f0bdb53e475f95c176dad316c220 P 1186876c38d64aa4b21d79b8ff6529e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ca7821dd7c144ddbb92ad9311b50970" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 37537 } } peers { permanent_uuid: "bfb0bf9113574a099141f94538e51f32" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 36961 } } peers { permanent_uuid: "1186876c38d64aa4b21d79b8ff6529e5" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37291 } }
I20260924 00:56:26.953192 13607 raft_consensus.cc:374] T 90c8f0bdb53e475f95c176dad316c220 P 1186876c38d64aa4b21d79b8ff6529e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:26.953286 13607 raft_consensus.cc:729] T 90c8f0bdb53e475f95c176dad316c220 P 1186876c38d64aa4b21d79b8ff6529e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1186876c38d64aa4b21d79b8ff6529e5, State: Initialized, Role: FOLLOWER
I20260924 00:56:26.953859 13607 consensus_queue.cc:260] T 90c8f0bdb53e475f95c176dad316c220 P 1186876c38d64aa4b21d79b8ff6529e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ca7821dd7c144ddbb92ad9311b50970" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 37537 } } peers { permanent_uuid: "bfb0bf9113574a099141f94538e51f32" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 36961 } } peers { permanent_uuid: "1186876c38d64aa4b21d79b8ff6529e5" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37291 } }
I20260924 00:56:26.953759 13606 raft_consensus.cc:348] T 90c8f0bdb53e475f95c176dad316c220 P 6ca7821dd7c144ddbb92ad9311b50970 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ca7821dd7c144ddbb92ad9311b50970" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 37537 } } peers { permanent_uuid: "bfb0bf9113574a099141f94538e51f32" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 36961 } } peers { permanent_uuid: "1186876c38d64aa4b21d79b8ff6529e5" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37291 } }
I20260924 00:56:26.954313 13606 raft_consensus.cc:374] T 90c8f0bdb53e475f95c176dad316c220 P 6ca7821dd7c144ddbb92ad9311b50970 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:26.954473 13606 raft_consensus.cc:729] T 90c8f0bdb53e475f95c176dad316c220 P 6ca7821dd7c144ddbb92ad9311b50970 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ca7821dd7c144ddbb92ad9311b50970, State: Initialized, Role: FOLLOWER
I20260924 00:56:26.955097 13606 consensus_queue.cc:260] T 90c8f0bdb53e475f95c176dad316c220 P 6ca7821dd7c144ddbb92ad9311b50970 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ca7821dd7c144ddbb92ad9311b50970" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 37537 } } peers { permanent_uuid: "bfb0bf9113574a099141f94538e51f32" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 36961 } } peers { permanent_uuid: "1186876c38d64aa4b21d79b8ff6529e5" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37291 } }
I20260924 00:56:26.960952 13324 heartbeater.cc:503] Master 127.8.64.190:34099 was elected leader, sending a full tablet report...
I20260924 00:56:26.961455 13607 ts_tablet_manager.cc:1428] T 90c8f0bdb53e475f95c176dad316c220 P 1186876c38d64aa4b21d79b8ff6529e5: Time spent starting tablet: real 0.020s user 0.009s sys 0.013s
I20260924 00:56:26.961453 13588 heartbeater.cc:503] Master 127.8.64.190:34099 was elected leader, sending a full tablet report...
I20260924 00:56:26.962236 13606 ts_tablet_manager.cc:1428] T 90c8f0bdb53e475f95c176dad316c220 P 6ca7821dd7c144ddbb92ad9311b50970: Time spent starting tablet: real 0.021s user 0.019s sys 0.000s
W20260924 00:56:26.979385 13325 tablet.cc:2369] T 90c8f0bdb53e475f95c176dad316c220 P 1186876c38d64aa4b21d79b8ff6529e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 00:56:27.095129 13589 tablet.cc:2369] T 90c8f0bdb53e475f95c176dad316c220 P 6ca7821dd7c144ddbb92ad9311b50970: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:56:27.132428 13612 raft_consensus.cc:482] T 90c8f0bdb53e475f95c176dad316c220 P bfb0bf9113574a099141f94538e51f32 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:56:27.132722 13612 raft_consensus.cc:504] T 90c8f0bdb53e475f95c176dad316c220 P bfb0bf9113574a099141f94538e51f32 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ca7821dd7c144ddbb92ad9311b50970" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 37537 } } peers { permanent_uuid: "bfb0bf9113574a099141f94538e51f32" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 36961 } } peers { permanent_uuid: "1186876c38d64aa4b21d79b8ff6529e5" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37291 } }
I20260924 00:56:27.134382 13612 leader_election.cc:288] T 90c8f0bdb53e475f95c176dad316c220 P bfb0bf9113574a099141f94538e51f32 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ca7821dd7c144ddbb92ad9311b50970 (127.8.64.131:37537), 1186876c38d64aa4b21d79b8ff6529e5 (127.8.64.129:37291)
I20260924 00:56:27.137603 13601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:27.134749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:60973 (local address 127.8.64.131:37537)
0924 00:56:27.135233 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:27.135242 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:27.135545 (+ 303us) server_negotiation.cc:407] Connection header received
0924 00:56:27.135740 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:27.135748 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:27.135814 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:27.135920 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:27.136969 (+ 1049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:27.136975 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:27.136980 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:56:27.137000 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:27.137034 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:56:27.137044 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:27.137172 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:27.137247 (+ 75us) server_negotiation.cc:299] Negotiation successful
0924 00:56:27.137333 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292}
I20260924 00:56:27.137840 13605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:27.134801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:50693 (local address 127.8.64.129:37291)
0924 00:56:27.135024 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:27.135032 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:27.135875 (+ 843us) server_negotiation.cc:407] Connection header received
0924 00:56:27.136082 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:27.136091 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:27.136137 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:27.136248 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:27.137160 (+ 912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:27.137165 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:27.137169 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:56:27.137186 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:27.137225 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:56:27.137236 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:27.137354 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:27.137479 (+ 125us) server_negotiation.cc:299] Negotiation successful
0924 00:56:27.137540 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260924 00:56:27.137866 13615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:27.134573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.131:37537 (local address 127.8.64.130:60973)
0924 00:56:27.135432 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:27.135451 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:27.135613 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:27.135944 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:27.135949 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:27.135973 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:27.136733 (+ 760us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 00:56:27.136737 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:27.136767 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 00:56:27.136819 (+ 52us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:27.137135 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:27.137140 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:27.137145 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 00:56:27.137288 (+ 143us) client_negotiation.cc:240] Negotiation successful
0924 00:56:27.137333 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":704,"thread_start_us":506,"threads_started":1}
I20260924 00:56:27.138031 13616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:27.134797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:37291 (local address 127.8.64.130:50693)
0924 00:56:27.135752 (+ 955us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:27.135781 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:27.135906 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:27.136269 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:27.136274 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:27.136297 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:27.136902 (+ 605us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 00:56:27.136904 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:27.136955 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 00:56:27.137021 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:27.137331 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:27.137336 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:27.137341 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 00:56:27.137521 (+ 180us) client_negotiation.cc:240] Negotiation successful
0924 00:56:27.137566 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":846,"thread_start_us":632,"threads_started":1}
I20260924 00:56:27.139047 13542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90c8f0bdb53e475f95c176dad316c220" candidate_uuid: "bfb0bf9113574a099141f94538e51f32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ca7821dd7c144ddbb92ad9311b50970" is_pre_election: true
I20260924 00:56:27.139047 13279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90c8f0bdb53e475f95c176dad316c220" candidate_uuid: "bfb0bf9113574a099141f94538e51f32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1186876c38d64aa4b21d79b8ff6529e5" is_pre_election: true
I20260924 00:56:27.139598 13279 raft_consensus.cc:2454] T 90c8f0bdb53e475f95c176dad316c220 P 1186876c38d64aa4b21d79b8ff6529e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfb0bf9113574a099141f94538e51f32 in term 0.
I20260924 00:56:27.139598 13542 raft_consensus.cc:2454] T 90c8f0bdb53e475f95c176dad316c220 P 6ca7821dd7c144ddbb92ad9311b50970 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfb0bf9113574a099141f94538e51f32 in term 0.
I20260924 00:56:27.140285 13345 leader_election.cc:302] T 90c8f0bdb53e475f95c176dad316c220 P bfb0bf9113574a099141f94538e51f32 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ca7821dd7c144ddbb92ad9311b50970, bfb0bf9113574a099141f94538e51f32; no voters:
I20260924 00:56:27.140693 13612 raft_consensus.cc:2781] T 90c8f0bdb53e475f95c176dad316c220 P bfb0bf9113574a099141f94538e51f32 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:56:27.140852 13612 raft_consensus.cc:482] T 90c8f0bdb53e475f95c176dad316c220 P bfb0bf9113574a099141f94538e51f32 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:56:27.140967 13612 raft_consensus.cc:3037] T 90c8f0bdb53e475f95c176dad316c220 P bfb0bf9113574a099141f94538e51f32 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:27.143222 13612 raft_consensus.cc:504] T 90c8f0bdb53e475f95c176dad316c220 P bfb0bf9113574a099141f94538e51f32 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ca7821dd7c144ddbb92ad9311b50970" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 37537 } } peers { permanent_uuid: "bfb0bf9113574a099141f94538e51f32" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 36961 } } peers { permanent_uuid: "1186876c38d64aa4b21d79b8ff6529e5" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37291 } }
I20260924 00:56:27.143774 13612 leader_election.cc:288] T 90c8f0bdb53e475f95c176dad316c220 P bfb0bf9113574a099141f94538e51f32 [CANDIDATE]: Term 1 election: Requested vote from peers 6ca7821dd7c144ddbb92ad9311b50970 (127.8.64.131:37537), 1186876c38d64aa4b21d79b8ff6529e5 (127.8.64.129:37291)
I20260924 00:56:27.144258 13279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90c8f0bdb53e475f95c176dad316c220" candidate_uuid: "bfb0bf9113574a099141f94538e51f32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1186876c38d64aa4b21d79b8ff6529e5"
I20260924 00:56:27.144236 13542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90c8f0bdb53e475f95c176dad316c220" candidate_uuid: "bfb0bf9113574a099141f94538e51f32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ca7821dd7c144ddbb92ad9311b50970"
I20260924 00:56:27.144469 13279 raft_consensus.cc:3037] T 90c8f0bdb53e475f95c176dad316c220 P 1186876c38d64aa4b21d79b8ff6529e5 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:27.144469 13542 raft_consensus.cc:3037] T 90c8f0bdb53e475f95c176dad316c220 P 6ca7821dd7c144ddbb92ad9311b50970 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:27.147678 13542 raft_consensus.cc:2454] T 90c8f0bdb53e475f95c176dad316c220 P 6ca7821dd7c144ddbb92ad9311b50970 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfb0bf9113574a099141f94538e51f32 in term 1.
I20260924 00:56:27.147678 13279 raft_consensus.cc:2454] T 90c8f0bdb53e475f95c176dad316c220 P 1186876c38d64aa4b21d79b8ff6529e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfb0bf9113574a099141f94538e51f32 in term 1.
I20260924 00:56:27.148096 13344 leader_election.cc:302] T 90c8f0bdb53e475f95c176dad316c220 P bfb0bf9113574a099141f94538e51f32 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1186876c38d64aa4b21d79b8ff6529e5, bfb0bf9113574a099141f94538e51f32; no voters:
I20260924 00:56:27.148416 13612 raft_consensus.cc:2781] T 90c8f0bdb53e475f95c176dad316c220 P bfb0bf9113574a099141f94538e51f32 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:56:27.148862 13612 raft_consensus.cc:686] T 90c8f0bdb53e475f95c176dad316c220 P bfb0bf9113574a099141f94538e51f32 [term 1 LEADER]: Becoming Leader. State: Replica: bfb0bf9113574a099141f94538e51f32, State: Running, Role: LEADER
I20260924 00:56:27.149354 13612 consensus_queue.cc:237] T 90c8f0bdb53e475f95c176dad316c220 P bfb0bf9113574a099141f94538e51f32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ca7821dd7c144ddbb92ad9311b50970" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 37537 } } peers { permanent_uuid: "bfb0bf9113574a099141f94538e51f32" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 36961 } } peers { permanent_uuid: "1186876c38d64aa4b21d79b8ff6529e5" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37291 } }
I20260924 00:56:27.154388 13138 catalog_manager.cc:5495] T 90c8f0bdb53e475f95c176dad316c220 P bfb0bf9113574a099141f94538e51f32 reported cstate change: term changed from 0 to 1, leader changed from <none> to bfb0bf9113574a099141f94538e51f32 (127.8.64.130). New cstate: current_term: 1 leader_uuid: "bfb0bf9113574a099141f94538e51f32" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ca7821dd7c144ddbb92ad9311b50970" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 37537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfb0bf9113574a099141f94538e51f32" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 36961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1186876c38d64aa4b21d79b8ff6529e5" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37291 } health_report { overall_health: UNKNOWN } } }
W20260924 00:56:27.163950 13458 tablet.cc:2369] T 90c8f0bdb53e475f95c176dad316c220 P bfb0bf9113574a099141f94538e51f32: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:56:27.195500 13604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:27.193170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36342 (local address 127.8.64.130:36961)
0924 00:56:27.193352 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:27.193358 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:27.193402 (+ 44us) server_negotiation.cc:407] Connection header received
0924 00:56:27.193656 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:27.193660 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:27.193760 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:27.193882 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:27.194883 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:27.194889 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:27.194894 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:56:27.194911 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:27.194949 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:56:27.194960 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:27.195084 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:27.195165 (+ 81us) server_negotiation.cc:299] Negotiation successful
0924 00:56:27.195222 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260924 00:56:27.203363 13279 raft_consensus.cc:1261] T 90c8f0bdb53e475f95c176dad316c220 P 1186876c38d64aa4b21d79b8ff6529e5 [term 1 FOLLOWER]: Refusing update from remote peer bfb0bf9113574a099141f94538e51f32: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:56:27.203406 13542 raft_consensus.cc:1261] T 90c8f0bdb53e475f95c176dad316c220 P 6ca7821dd7c144ddbb92ad9311b50970 [term 1 FOLLOWER]: Refusing update from remote peer bfb0bf9113574a099141f94538e51f32: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:56:27.204253 13617 consensus_queue.cc:1035] T 90c8f0bdb53e475f95c176dad316c220 P bfb0bf9113574a099141f94538e51f32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ca7821dd7c144ddbb92ad9311b50970" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 37537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:56:27.204609 13612 consensus_queue.cc:1035] T 90c8f0bdb53e475f95c176dad316c220 P bfb0bf9113574a099141f94538e51f32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1186876c38d64aa4b21d79b8ff6529e5" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:56:27.218050 13620 mvcc.cc:205] Tried to move back new op lower bound from 7332705841975439360 to 7332705841768677376. Current Snapshot: MvccSnapshot[applied={T|T < 7332705841975439360 or (T in {7332705841975439360})}]
I20260924 00:56:27.237032 13138 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:56:27.237322 13138 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:56:27.240643 13138 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b42291201c374266930125689ef020a9: Sending DeleteTablet for 3 replicas of tablet 90c8f0bdb53e475f95c176dad316c220
I20260924 00:56:27.241725 13522 tablet_service.cc:1514] Processing DeleteTablet for tablet 90c8f0bdb53e475f95c176dad316c220 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:56:27 UTC) from {username='slave'} at 127.0.0.1:59338
I20260924 00:56:27.241724 13392 tablet_service.cc:1514] Processing DeleteTablet for tablet 90c8f0bdb53e475f95c176dad316c220 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:56:27 UTC) from {username='slave'} at 127.0.0.1:36336
I20260924 00:56:27.242282 13259 tablet_service.cc:1514] Processing DeleteTablet for tablet 90c8f0bdb53e475f95c176dad316c220 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:56:27 UTC) from {username='slave'} at 127.0.0.1:56714
I20260924 00:56:27.244594 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 13197
I20260924 00:56:27.253636 13632 tablet_replica.cc:330] T 90c8f0bdb53e475f95c176dad316c220 P 1186876c38d64aa4b21d79b8ff6529e5: stopping tablet replica
I20260924 00:56:27.254194 13632 raft_consensus.cc:2229] T 90c8f0bdb53e475f95c176dad316c220 P 1186876c38d64aa4b21d79b8ff6529e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:56:27.254832 13632 raft_consensus.cc:2258] T 90c8f0bdb53e475f95c176dad316c220 P 1186876c38d64aa4b21d79b8ff6529e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:56:27.255759 13630 tablet_replica.cc:330] T 90c8f0bdb53e475f95c176dad316c220 P bfb0bf9113574a099141f94538e51f32: stopping tablet replica
I20260924 00:56:27.256280 13630 raft_consensus.cc:2229] T 90c8f0bdb53e475f95c176dad316c220 P bfb0bf9113574a099141f94538e51f32 [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:56:27.257030 13630 raft_consensus.cc:2258] T 90c8f0bdb53e475f95c176dad316c220 P bfb0bf9113574a099141f94538e51f32 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:56:27.257503 13605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:27.246566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56720 (local address 127.8.64.129:37291)
0924 00:56:27.246732 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:27.246738 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:27.251150 (+ 4412us) server_negotiation.cc:407] Connection header received
0924 00:56:27.251373 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:27.251379 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:27.251485 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:27.251588 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:27.256692 (+ 5104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:27.256698 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:27.256702 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:56:27.256724 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:27.256767 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:56:27.256783 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:27.256874 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:27.257138 (+ 264us) server_negotiation.cc:299] Negotiation successful
0924 00:56:27.257200 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260924 00:56:27.258752 13631 tablet_replica.cc:330] T 90c8f0bdb53e475f95c176dad316c220 P 6ca7821dd7c144ddbb92ad9311b50970: stopping tablet replica
I20260924 00:56:27.258817 13319 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:56:27.259115 13631 raft_consensus.cc:2229] T 90c8f0bdb53e475f95c176dad316c220 P 6ca7821dd7c144ddbb92ad9311b50970 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:56:27.260123 13631 raft_consensus.cc:2258] T 90c8f0bdb53e475f95c176dad316c220 P 6ca7821dd7c144ddbb92ad9311b50970 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:56:27.261121 13630 ts_tablet_manager.cc:1905] T 90c8f0bdb53e475f95c176dad316c220 P bfb0bf9113574a099141f94538e51f32: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:56:27.262063 13631 ts_tablet_manager.cc:1905] T 90c8f0bdb53e475f95c176dad316c220 P 6ca7821dd7c144ddbb92ad9311b50970: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:56:27.269404 13630 ts_tablet_manager.cc:1918] T 90c8f0bdb53e475f95c176dad316c220 P bfb0bf9113574a099141f94538e51f32: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:56:27.269572 13630 log.cc:1196] T 90c8f0bdb53e475f95c176dad316c220 P bfb0bf9113574a099141f94538e51f32: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-1/wal/wals/90c8f0bdb53e475f95c176dad316c220
I20260924 00:56:27.269659 13631 ts_tablet_manager.cc:1918] T 90c8f0bdb53e475f95c176dad316c220 P 6ca7821dd7c144ddbb92ad9311b50970: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:56:27.269824 13631 log.cc:1196] T 90c8f0bdb53e475f95c176dad316c220 P 6ca7821dd7c144ddbb92ad9311b50970: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790211343099518-8450-0/minicluster-data/ts-2/wal/wals/90c8f0bdb53e475f95c176dad316c220
I20260924 00:56:27.270200 13630 ts_tablet_manager.cc:1939] T 90c8f0bdb53e475f95c176dad316c220 P bfb0bf9113574a099141f94538e51f32: Deleting consensus metadata
I20260924 00:56:27.270337 13631 ts_tablet_manager.cc:1939] T 90c8f0bdb53e475f95c176dad316c220 P 6ca7821dd7c144ddbb92ad9311b50970: Deleting consensus metadata
I20260924 00:56:27.272018 13122 catalog_manager.cc:4841] TS bfb0bf9113574a099141f94538e51f32 (127.8.64.130:36961): tablet 90c8f0bdb53e475f95c176dad316c220 (table test-table [id=5f896c1213b64d0a897fb7e37b1acd71]) successfully deleted
I20260924 00:56:27.272131 13124 catalog_manager.cc:4841] TS 6ca7821dd7c144ddbb92ad9311b50970 (127.8.64.131:37537): tablet 90c8f0bdb53e475f95c176dad316c220 (table test-table [id=5f896c1213b64d0a897fb7e37b1acd71]) successfully deleted
I20260924 00:56:27.259416 13632 ts_tablet_manager.cc:1905] T 90c8f0bdb53e475f95c176dad316c220 P 1186876c38d64aa4b21d79b8ff6529e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:56:27.377437 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 13197
W20260924 00:56:27.394708 13123 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.64.129:37291 (error 108)
I20260924 00:56:27.394999 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 13328
I20260924 00:56:27.396776 13603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:27.396288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:37291 (local address 127.0.0.1:56732)
0924 00:56:27.396474 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:27.396610 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.129:37291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260924 00:56:27.397027 13123 catalog_manager.cc:4584] TS 1186876c38d64aa4b21d79b8ff6529e5 (127.8.64.129:37291): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 90c8f0bdb53e475f95c176dad316c220: Network error: Client connection negotiation failed: client connection to 127.8.64.129:37291: connect: Connection refused (error 111)
I20260924 00:56:27.398047 13604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:27.395858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36354 (local address 127.8.64.130:36961)
0924 00:56:27.396015 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:27.396019 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:27.396087 (+ 68us) server_negotiation.cc:407] Connection header received
0924 00:56:27.396302 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:27.396309 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:27.396376 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:27.396481 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:27.397405 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:27.397411 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:27.397415 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:56:27.397442 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:27.397473 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:56:27.397484 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:27.397628 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:27.397727 (+ 99us) server_negotiation.cc:299] Negotiation successful
0924 00:56:27.397783 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260924 00:56:27.398895 13452 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:56:27.450728 13603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:27.450279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:37291 (local address 127.0.0.1:56734)
0924 00:56:27.450493 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:27.450594 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.129:37291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260924 00:56:27.451711 13603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:27.451368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:37291 (local address 127.0.0.1:56746)
0924 00:56:27.451520 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:27.451601 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.129:37291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260924 00:56:27.508819 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 13328
I20260924 00:56:27.512159 13603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:27.511734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:37291 (local address 127.0.0.1:56760)
0924 00:56:27.511927 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:27.512025 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.129:37291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260924 00:56:27.513217 13603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:27.512841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:37291 (local address 127.0.0.1:56772)
0924 00:56:27.513009 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:27.513088 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.129:37291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260924 00:56:27.527433 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 13460
I20260924 00:56:27.530167 13601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:27.528103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59342 (local address 127.8.64.131:37537)
0924 00:56:27.528278 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:27.528284 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:27.528327 (+ 43us) server_negotiation.cc:407] Connection header received
0924 00:56:27.528558 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:27.528563 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:27.528646 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:27.528756 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:27.529536 (+ 780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:27.529541 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:27.529554 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:56:27.529577 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:27.529607 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:56:27.529618 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:27.529748 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:27.529827 (+ 79us) server_negotiation.cc:299] Negotiation successful
0924 00:56:27.529886 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260924 00:56:27.530774 13583 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:56:27.593688 13603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:27.593191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:37291 (local address 127.0.0.1:56774)
0924 00:56:27.593418 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:27.593522 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.129:37291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260924 00:56:27.594789 13603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:27.594380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:37291 (local address 127.0.0.1:56776)
0924 00:56:27.594587 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:27.594664 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.129:37291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260924 00:56:27.618881 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 13460
I20260924 00:56:27.635035 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 13106
I20260924 00:56:27.636168 13168 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:56:27.745648 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 13106
2026-09-24T00:56:27Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2793 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260924 00:56:27.786378 8450 mini_oidc.cc:152] Starting JWKS server
I20260924 00:56:27.788757 8450 webserver.cc:460] Webserver started at https://127.0.0.1:34085/ using document root <none> and password file <none>
I20260924 00:56:27.789017 8450 mini_oidc.cc:189] Starting OIDC Discovery server
I20260924 00:56:27.789588 8450 webserver.cc:460] Webserver started at http://127.0.0.1:36391/ using document root <none> and password file <none>
2026-09-24T00:56:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:56:27Z Disabled control of system clock
I20260924 00:56:27.818109 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:34537
--webserver_interface=127.8.64.190
--webserver_port=0
--builtin_ntp_servers=127.8.64.148:36091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:34537
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34085/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790211343099518-8450-0/testchainca.cert with env {}
W20260924 00:56:28.134342 13644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:28.134673 13644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:28.134833 13644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:28.142261 13644 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:56:28.142452 13644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:28.142552 13644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:28.142663 13644 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:56:28.142733 13644 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260924 00:56:28.142823 13644 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260924 00:56:28.142895 13644 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34085/jwks.json
I20260924 00:56:28.152840 13644 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:36091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:34537
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:34537
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34085/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790211343099518-8450-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:28.154160 13644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:28.155448 13644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:28.167066 13650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:28.167037 13652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:28.167340 13649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:28.167789 13644 server_base.cc:1034] running on GCE node
I20260924 00:56:28.168515 13644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:28.169835 13644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:28.171053 13644 hybrid_clock.cc:648] HybridClock initialized: now 1790211388171019 us; error 46 us; skew 500 ppm
I20260924 00:56:28.171401 13644 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 00:56:28.173617 13644 webserver.cc:460] Webserver started at http://127.8.64.190:46091/ using document root <none> and password file <none>
I20260924 00:56:28.174283 13644 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:28.174484 13644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:28.174789 13644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:56:28.177547 13644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "8ac516dd11104cb0827bcec9d83cef49"
format_stamp: "Formatted at 2026-09-24 00:56:28 on dist-test-slave-mmhs"
server_key: "46013d8b1ebb50628123236757a8fec5"
server_key_iv: "97899df91b9dc46b55d69772c2d33dc9"
server_key_version: "encryptionkey@0"
I20260924 00:56:28.178375 13644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "8ac516dd11104cb0827bcec9d83cef49"
format_stamp: "Formatted at 2026-09-24 00:56:28 on dist-test-slave-mmhs"
server_key: "46013d8b1ebb50628123236757a8fec5"
server_key_iv: "97899df91b9dc46b55d69772c2d33dc9"
server_key_version: "encryptionkey@0"
I20260924 00:56:28.183411 13644 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260924 00:56:28.186740 13659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:28.188186 13644 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260924 00:56:28.188367 13644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "8ac516dd11104cb0827bcec9d83cef49"
format_stamp: "Formatted at 2026-09-24 00:56:28 on dist-test-slave-mmhs"
server_key: "46013d8b1ebb50628123236757a8fec5"
server_key_iv: "97899df91b9dc46b55d69772c2d33dc9"
server_key_version: "encryptionkey@0"
I20260924 00:56:28.188584 13644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:28.214114 13644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:28.215005 13644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:28.215324 13644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:28.233585 13644 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.190:34537
I20260924 00:56:28.233606 13710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.190:34537 every 8 connection(s)
I20260924 00:56:28.234910 13644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 00:56:28.235620 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 13644
I20260924 00:56:28.235896 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 00:56:28.236657 8450 external_mini_cluster.cc:1428] Setting key 6c2b17a134917a48ab09094d7d82d4ef
I20260924 00:56:28.239357 13711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:28.249660 13714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:28.238497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51554 (local address 127.8.64.190:34537)
0924 00:56:28.239521 (+ 1024us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:28.239542 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:28.239584 (+ 42us) server_negotiation.cc:407] Connection header received
0924 00:56:28.241165 (+ 1581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:28.241186 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:28.241638 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:28.242023 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:28.242626 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:28.243916 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:28.245332 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:28.246052 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:28.246248 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:28.247441 (+ 1193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:28.247449 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:28.247464 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:56:28.248014 (+ 550us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:28.248084 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:56:28.248107 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:28.248257 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:28.248489 (+ 232us) server_negotiation.cc:299] Negotiation successful
0924 00:56:28.248779 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":350,"threads_started":1}
I20260924 00:56:28.249996 13711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ac516dd11104cb0827bcec9d83cef49: Bootstrap starting.
I20260924 00:56:28.253319 8450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 00:56:28.253479 13711 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ac516dd11104cb0827bcec9d83cef49: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:28.254500 13711 log.cc:824] T 00000000000000000000000000000000 P 8ac516dd11104cb0827bcec9d83cef49: Log is configured to *not* fsync() on all Append() calls
I20260924 00:56:28.257164 13711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ac516dd11104cb0827bcec9d83cef49: No bootstrap required, opened a new log
I20260924 00:56:28.264161 13711 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ac516dd11104cb0827bcec9d83cef49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ac516dd11104cb0827bcec9d83cef49" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 34537 } }
I20260924 00:56:28.264539 13711 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ac516dd11104cb0827bcec9d83cef49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:28.264647 13711 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ac516dd11104cb0827bcec9d83cef49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ac516dd11104cb0827bcec9d83cef49, State: Initialized, Role: FOLLOWER
I20260924 00:56:28.265257 13711 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ac516dd11104cb0827bcec9d83cef49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ac516dd11104cb0827bcec9d83cef49" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 34537 } }
I20260924 00:56:28.265503 13711 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ac516dd11104cb0827bcec9d83cef49 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:56:28.265645 13711 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ac516dd11104cb0827bcec9d83cef49 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:56:28.265834 13711 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ac516dd11104cb0827bcec9d83cef49 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:28.267970 13711 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ac516dd11104cb0827bcec9d83cef49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ac516dd11104cb0827bcec9d83cef49" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 34537 } }
I20260924 00:56:28.268468 13711 leader_election.cc:302] T 00000000000000000000000000000000 P 8ac516dd11104cb0827bcec9d83cef49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ac516dd11104cb0827bcec9d83cef49; no voters:
I20260924 00:56:28.269052 13711 leader_election.cc:288] T 00000000000000000000000000000000 P 8ac516dd11104cb0827bcec9d83cef49 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:56:28.269234 13716 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ac516dd11104cb0827bcec9d83cef49 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:56:28.270638 13716 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ac516dd11104cb0827bcec9d83cef49 [term 1 LEADER]: Becoming Leader. State: Replica: 8ac516dd11104cb0827bcec9d83cef49, State: Running, Role: LEADER
I20260924 00:56:28.271207 13711 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ac516dd11104cb0827bcec9d83cef49 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:56:28.271143 13716 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ac516dd11104cb0827bcec9d83cef49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ac516dd11104cb0827bcec9d83cef49" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 34537 } }
I20260924 00:56:28.274233 13718 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ac516dd11104cb0827bcec9d83cef49 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ac516dd11104cb0827bcec9d83cef49. Latest consensus state: current_term: 1 leader_uuid: "8ac516dd11104cb0827bcec9d83cef49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ac516dd11104cb0827bcec9d83cef49" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 34537 } } }
I20260924 00:56:28.274257 13717 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ac516dd11104cb0827bcec9d83cef49 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ac516dd11104cb0827bcec9d83cef49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ac516dd11104cb0827bcec9d83cef49" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 34537 } } }
I20260924 00:56:28.274744 13718 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ac516dd11104cb0827bcec9d83cef49 [sys.catalog]: This master's current role is: LEADER
I20260924 00:56:28.274740 13717 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ac516dd11104cb0827bcec9d83cef49 [sys.catalog]: This master's current role is: LEADER
I20260924 00:56:28.275705 13724 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:56:28.283381 13724 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:56:28.294027 13724 catalog_manager.cc:1347] Generated new cluster ID: b6c546ec15b04151be23a1e7a625a885
I20260924 00:56:28.294148 13724 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:56:28.323448 13724 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:56:28.324703 13724 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:56:28.337483 13724 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ac516dd11104cb0827bcec9d83cef49: Generated new TSK 0
I20260924 00:56:28.338400 13724 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260924 00:56:28.378903 13741 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260924 00:56:28.379078 13741 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260924 00:56:28.380174 13714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:28.358449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51568 (local address 127.8.64.190:34537)
0924 00:56:28.358638 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:28.358644 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:28.359109 (+ 465us) server_negotiation.cc:407] Connection header received
0924 00:56:28.359382 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:28.359388 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:28.359486 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:28.359641 (+ 155us) server_negotiation.cc:226] Negotiated authn=JWT
0924 00:56:28.360130 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:28.361325 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:28.362794 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:28.363518 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:28.363636 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:28.364160 (+ 524us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0924 00:56:28.379519 (+ 15359us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34085/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0924 00:56:28.379850 (+ 331us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51568: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34085/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":56}
W20260924 00:56:28.380298 13714 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51568: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34085/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260924 00:56:28.381456 13740 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:56:28.358260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:34537 (local address 127.0.0.1:51568)
0924 00:56:28.359009 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:28.359025 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:28.359165 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:28.359668 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:28.359671 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:28.359680 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0924 00:56:28.359980 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:28.359987 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:28.361472 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:28.361476 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:28.362663 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:28.362674 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:28.363687 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:28.363691 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:28.363815 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:28.363884 (+ 69us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0924 00:56:28.379794 (+ 15910us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34085/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0924 00:56:28.380041 (+ 247us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.64.190:34537: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34085/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":436,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.8.64.190:34537: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34085/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260924 00:56:28.383091 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 13644
I20260924 00:56:28.384317 13706 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:56:28.480605 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 13644
2026-09-24T00:56:28Z chronyd exiting
I20260924 00:56:28.530727 8450 test_util.cc:175] -----------------------------------------------
I20260924 00:56:28.530864 8450 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790211343099518-8450-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (748 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260924 00:56:28.534554 8450 mini_oidc.cc:152] Starting JWKS server
I20260924 00:56:28.536355 8450 webserver.cc:460] Webserver started at https://127.0.0.1:46427/ using document root <none> and password file <none>
I20260924 00:56:28.536578 8450 mini_oidc.cc:189] Starting OIDC Discovery server
I20260924 00:56:28.537184 8450 webserver.cc:460] Webserver started at http://127.0.0.1:43575/ using document root <none> and password file <none>
2026-09-24T00:56:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:56:28Z Disabled control of system clock
I20260924 00:56:28.563437 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:39395
--webserver_interface=127.8.64.190
--webserver_port=0
--builtin_ntp_servers=127.8.64.148:37285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:39395
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46427/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790211343099518-8450-0/testchainca.cert with env {}
W20260924 00:56:28.878093 13748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:28.878394 13748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:28.878484 13748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:28.885450 13748 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:56:28.885552 13748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:28.885589 13748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:28.885645 13748 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:56:28.885689 13748 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260924 00:56:28.885743 13748 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260924 00:56:28.885783 13748 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46427/jwks.json
I20260924 00:56:28.895341 13748 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:37285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:39395
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:39395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46427/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790211343099518-8450-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:28.896562 13748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:28.897785 13748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:28.909948 13756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:28.909909 13754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:28.909992 13753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:28.915926 13748 server_base.cc:1034] running on GCE node
I20260924 00:56:28.916787 13748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:28.918185 13748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:28.919386 13748 hybrid_clock.cc:648] HybridClock initialized: now 1790211388919321 us; error 58 us; skew 500 ppm
I20260924 00:56:28.919744 13748 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 00:56:28.927939 13748 webserver.cc:460] Webserver started at http://127.8.64.190:36063/ using document root <none> and password file <none>
I20260924 00:56:28.928684 13748 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:28.928824 13748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:28.929162 13748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:56:28.932231 13748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "04bdaa45f9bd40a593bf59276d2f1b3e"
format_stamp: "Formatted at 2026-09-24 00:56:28 on dist-test-slave-mmhs"
server_key: "2c070313ef7e0925a10ef8dc46c8ba47"
server_key_iv: "625d560090eaeb1cc0a168ccb5757d2b"
server_key_version: "encryptionkey@0"
I20260924 00:56:28.933019 13748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "04bdaa45f9bd40a593bf59276d2f1b3e"
format_stamp: "Formatted at 2026-09-24 00:56:28 on dist-test-slave-mmhs"
server_key: "2c070313ef7e0925a10ef8dc46c8ba47"
server_key_iv: "625d560090eaeb1cc0a168ccb5757d2b"
server_key_version: "encryptionkey@0"
I20260924 00:56:28.938261 13748 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260924 00:56:28.941710 13762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:28.943074 13748 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 00:56:28.943241 13748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "04bdaa45f9bd40a593bf59276d2f1b3e"
format_stamp: "Formatted at 2026-09-24 00:56:28 on dist-test-slave-mmhs"
server_key: "2c070313ef7e0925a10ef8dc46c8ba47"
server_key_iv: "625d560090eaeb1cc0a168ccb5757d2b"
server_key_version: "encryptionkey@0"
I20260924 00:56:28.943455 13748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:28.956681 13748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:28.957450 13748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:28.957755 13748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:28.975585 13748 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.190:39395
I20260924 00:56:28.975605 13814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.190:39395 every 8 connection(s)
I20260924 00:56:28.976926 13748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 00:56:28.980505 13815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:28.981628 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 13748
I20260924 00:56:28.981824 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 00:56:28.982482 8450 external_mini_cluster.cc:1428] Setting key 062d2939c554230f8b24d2f66ce2906d
I20260924 00:56:28.992630 13815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04bdaa45f9bd40a593bf59276d2f1b3e: Bootstrap starting.
I20260924 00:56:28.994982 13818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:28.984258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42272 (local address 127.8.64.190:39395)
0924 00:56:28.985244 (+ 986us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:28.985257 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:28.985298 (+ 41us) server_negotiation.cc:407] Connection header received
0924 00:56:28.986866 (+ 1568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:28.986888 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:28.987333 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:28.987717 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:28.988239 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:28.989451 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:28.990752 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:28.991395 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:28.991580 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:28.992698 (+ 1118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:28.992711 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:28.992724 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:56:28.993293 (+ 569us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:28.993367 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:56:28.993387 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:28.993507 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:28.993728 (+ 221us) server_negotiation.cc:299] Negotiation successful
0924 00:56:28.994022 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":353,"threads_started":1}
I20260924 00:56:28.996155 13815 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04bdaa45f9bd40a593bf59276d2f1b3e: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:28.997238 13815 log.cc:824] T 00000000000000000000000000000000 P 04bdaa45f9bd40a593bf59276d2f1b3e: Log is configured to *not* fsync() on all Append() calls
I20260924 00:56:28.998754 8450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 00:56:29.000082 13815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04bdaa45f9bd40a593bf59276d2f1b3e: No bootstrap required, opened a new log
I20260924 00:56:29.006879 13815 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04bdaa45f9bd40a593bf59276d2f1b3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04bdaa45f9bd40a593bf59276d2f1b3e" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 39395 } }
I20260924 00:56:29.007228 13815 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04bdaa45f9bd40a593bf59276d2f1b3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:29.007346 13815 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04bdaa45f9bd40a593bf59276d2f1b3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04bdaa45f9bd40a593bf59276d2f1b3e, State: Initialized, Role: FOLLOWER
I20260924 00:56:29.007957 13815 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04bdaa45f9bd40a593bf59276d2f1b3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04bdaa45f9bd40a593bf59276d2f1b3e" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 39395 } }
I20260924 00:56:29.008194 13815 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04bdaa45f9bd40a593bf59276d2f1b3e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:56:29.008337 13815 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04bdaa45f9bd40a593bf59276d2f1b3e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:56:29.008507 13815 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04bdaa45f9bd40a593bf59276d2f1b3e [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:29.010670 13815 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04bdaa45f9bd40a593bf59276d2f1b3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04bdaa45f9bd40a593bf59276d2f1b3e" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 39395 } }
I20260924 00:56:29.011145 13815 leader_election.cc:302] T 00000000000000000000000000000000 P 04bdaa45f9bd40a593bf59276d2f1b3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04bdaa45f9bd40a593bf59276d2f1b3e; no voters:
I20260924 00:56:29.011703 13815 leader_election.cc:288] T 00000000000000000000000000000000 P 04bdaa45f9bd40a593bf59276d2f1b3e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:56:29.011844 13820 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04bdaa45f9bd40a593bf59276d2f1b3e [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:56:29.012521 13820 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04bdaa45f9bd40a593bf59276d2f1b3e [term 1 LEADER]: Becoming Leader. State: Replica: 04bdaa45f9bd40a593bf59276d2f1b3e, State: Running, Role: LEADER
I20260924 00:56:29.013064 13820 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04bdaa45f9bd40a593bf59276d2f1b3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04bdaa45f9bd40a593bf59276d2f1b3e" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 39395 } }
I20260924 00:56:29.013818 13815 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04bdaa45f9bd40a593bf59276d2f1b3e [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:56:29.015861 13821 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04bdaa45f9bd40a593bf59276d2f1b3e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "04bdaa45f9bd40a593bf59276d2f1b3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04bdaa45f9bd40a593bf59276d2f1b3e" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 39395 } } }
I20260924 00:56:29.016331 13821 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04bdaa45f9bd40a593bf59276d2f1b3e [sys.catalog]: This master's current role is: LEADER
I20260924 00:56:29.017341 13822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04bdaa45f9bd40a593bf59276d2f1b3e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04bdaa45f9bd40a593bf59276d2f1b3e. Latest consensus state: current_term: 1 leader_uuid: "04bdaa45f9bd40a593bf59276d2f1b3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04bdaa45f9bd40a593bf59276d2f1b3e" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 39395 } } }
I20260924 00:56:29.017691 13822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04bdaa45f9bd40a593bf59276d2f1b3e [sys.catalog]: This master's current role is: LEADER
I20260924 00:56:29.018193 13829 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:56:29.022497 13829 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:56:29.031183 13829 catalog_manager.cc:1347] Generated new cluster ID: 97a560d88a024c14919f0d40b484e84e
I20260924 00:56:29.031312 13829 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:56:29.053319 13829 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:56:29.054659 13829 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:56:29.067848 13829 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04bdaa45f9bd40a593bf59276d2f1b3e: Generated new TSK 0
I20260924 00:56:29.068732 13829 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260924 00:56:29.109349 13845 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260924 00:56:29.109477 13845 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260924 00:56:29.110519 13818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:29.090442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42284 (local address 127.8.64.190:39395)
0924 00:56:29.090684 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:29.090690 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:29.091089 (+ 399us) server_negotiation.cc:407] Connection header received
0924 00:56:29.091299 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:29.091303 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:29.091381 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:29.091519 (+ 138us) server_negotiation.cc:226] Negotiated authn=JWT
0924 00:56:29.092059 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:29.092749 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:29.094776 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:29.095211 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:29.095306 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:29.095910 (+ 604us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0924 00:56:29.109935 (+ 14025us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46427/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0924 00:56:29.110232 (+ 297us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42284: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46427/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":66}
W20260924 00:56:29.110669 13818 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42284: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46427/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260924 00:56:29.110904 13844 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:56:29.090255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:39395 (local address 127.0.0.1:42284)
0924 00:56:29.090951 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:29.090970 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:29.091116 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:29.091564 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:29.091569 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:29.091582 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0924 00:56:29.091892 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:29.091903 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:29.092905 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:29.092911 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:29.094628 (+ 1717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:29.094642 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:29.095368 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:29.095375 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:29.095593 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:29.095639 (+ 46us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0924 00:56:29.110178 (+ 14539us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46427/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0924 00:56:29.110487 (+ 309us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.64.190:39395: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46427/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":344,"threads_started":1}
I20260924 00:56:29.112563 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 13748
I20260924 00:56:29.113785 13810 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:56:29.210584 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 13748
2026-09-24T00:56:29Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (730 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260924 00:56:29.265246 8450 mini_oidc.cc:152] Starting JWKS server
I20260924 00:56:29.267014 8450 webserver.cc:460] Webserver started at https://127.0.0.1:45499/ using document root <none> and password file <none>
I20260924 00:56:29.267222 8450 mini_oidc.cc:189] Starting OIDC Discovery server
I20260924 00:56:29.267766 8450 webserver.cc:460] Webserver started at http://127.0.0.1:46525/ using document root <none> and password file <none>
2026-09-24T00:56:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:56:29Z Disabled control of system clock
I20260924 00:56:29.295092 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:34687
--webserver_interface=127.8.64.190
--webserver_port=0
--builtin_ntp_servers=127.8.64.148:42859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:34687
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45499/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260924 00:56:29.616808 13852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:29.617120 13852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:29.617285 13852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:29.624498 13852 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:56:29.624605 13852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:29.624694 13852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:29.624801 13852 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:56:29.624881 13852 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260924 00:56:29.624975 13852 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260924 00:56:29.625052 13852 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45499/jwks.json
I20260924 00:56:29.634141 13852 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:42859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:34687
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:34687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45499/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:29.635463 13852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:29.637219 13852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:29.648648 13860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:29.648737 13858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:29.648726 13857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:29.649533 13852 server_base.cc:1034] running on GCE node
I20260924 00:56:29.650339 13852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:29.651722 13852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:29.652946 13852 hybrid_clock.cc:648] HybridClock initialized: now 1790211389652901 us; error 43 us; skew 500 ppm
I20260924 00:56:29.653293 13852 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 00:56:29.655551 13852 webserver.cc:460] Webserver started at http://127.8.64.190:42695/ using document root <none> and password file <none>
I20260924 00:56:29.656248 13852 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:29.656378 13852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:29.656698 13852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:56:29.659574 13852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "079b49d95f494f1da866b1c948053a20"
format_stamp: "Formatted at 2026-09-24 00:56:29 on dist-test-slave-mmhs"
server_key: "ae48cb4e31cb86c7b3e5fefa9d63b07e"
server_key_iv: "06078baaca58c27ec0a026c5eb2f6ba0"
server_key_version: "encryptionkey@0"
I20260924 00:56:29.660357 13852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "079b49d95f494f1da866b1c948053a20"
format_stamp: "Formatted at 2026-09-24 00:56:29 on dist-test-slave-mmhs"
server_key: "ae48cb4e31cb86c7b3e5fefa9d63b07e"
server_key_iv: "06078baaca58c27ec0a026c5eb2f6ba0"
server_key_version: "encryptionkey@0"
I20260924 00:56:29.665431 13852 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260924 00:56:29.668917 13866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:29.670260 13852 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260924 00:56:29.670511 13852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "079b49d95f494f1da866b1c948053a20"
format_stamp: "Formatted at 2026-09-24 00:56:29 on dist-test-slave-mmhs"
server_key: "ae48cb4e31cb86c7b3e5fefa9d63b07e"
server_key_iv: "06078baaca58c27ec0a026c5eb2f6ba0"
server_key_version: "encryptionkey@0"
I20260924 00:56:29.670730 13852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:29.685673 13852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:29.686573 13852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:29.686887 13852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:29.705566 13852 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.190:34687
I20260924 00:56:29.705672 13917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.190:34687 every 8 connection(s)
I20260924 00:56:29.706969 13852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 00:56:29.710883 13918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:29.713397 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 13852
I20260924 00:56:29.713610 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 00:56:29.714417 8450 external_mini_cluster.cc:1428] Setting key 8462e1641be1aced99cfd4d0b7499a54
I20260924 00:56:29.723276 13918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 079b49d95f494f1da866b1c948053a20: Bootstrap starting.
I20260924 00:56:29.727056 13918 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 079b49d95f494f1da866b1c948053a20: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:29.728368 13918 log.cc:824] T 00000000000000000000000000000000 P 079b49d95f494f1da866b1c948053a20: Log is configured to *not* fsync() on all Append() calls
I20260924 00:56:29.728893 13921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:29.716473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43498 (local address 127.8.64.190:34687)
0924 00:56:29.717582 (+ 1109us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:29.717594 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:29.717641 (+ 47us) server_negotiation.cc:407] Connection header received
0924 00:56:29.719452 (+ 1811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:29.719475 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:29.719982 (+ 507us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:29.720344 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:29.720900 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:29.722253 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:29.724066 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:29.724761 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:29.724908 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:29.726140 (+ 1232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:29.726148 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:29.726160 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:56:29.726949 (+ 789us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:29.727055 (+ 106us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:56:29.727077 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:29.727222 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:29.727478 (+ 256us) server_negotiation.cc:299] Negotiation successful
0924 00:56:29.727830 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":692,"thread_start_us":448,"threads_started":1}
I20260924 00:56:29.731601 13918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 079b49d95f494f1da866b1c948053a20: No bootstrap required, opened a new log
I20260924 00:56:29.732875 8450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 00:56:29.740720 13918 raft_consensus.cc:348] T 00000000000000000000000000000000 P 079b49d95f494f1da866b1c948053a20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "079b49d95f494f1da866b1c948053a20" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 34687 } }
I20260924 00:56:29.741134 13918 raft_consensus.cc:374] T 00000000000000000000000000000000 P 079b49d95f494f1da866b1c948053a20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:29.741257 13918 raft_consensus.cc:729] T 00000000000000000000000000000000 P 079b49d95f494f1da866b1c948053a20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 079b49d95f494f1da866b1c948053a20, State: Initialized, Role: FOLLOWER
I20260924 00:56:29.741869 13918 consensus_queue.cc:260] T 00000000000000000000000000000000 P 079b49d95f494f1da866b1c948053a20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "079b49d95f494f1da866b1c948053a20" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 34687 } }
I20260924 00:56:29.742120 13918 raft_consensus.cc:388] T 00000000000000000000000000000000 P 079b49d95f494f1da866b1c948053a20 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:56:29.742316 13918 raft_consensus.cc:482] T 00000000000000000000000000000000 P 079b49d95f494f1da866b1c948053a20 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:56:29.742509 13918 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 079b49d95f494f1da866b1c948053a20 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:29.744854 13918 raft_consensus.cc:504] T 00000000000000000000000000000000 P 079b49d95f494f1da866b1c948053a20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "079b49d95f494f1da866b1c948053a20" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 34687 } }
I20260924 00:56:29.745421 13918 leader_election.cc:302] T 00000000000000000000000000000000 P 079b49d95f494f1da866b1c948053a20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 079b49d95f494f1da866b1c948053a20; no voters:
I20260924 00:56:29.746109 13918 leader_election.cc:288] T 00000000000000000000000000000000 P 079b49d95f494f1da866b1c948053a20 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:56:29.746351 13923 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 079b49d95f494f1da866b1c948053a20 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:56:29.747107 13923 raft_consensus.cc:686] T 00000000000000000000000000000000 P 079b49d95f494f1da866b1c948053a20 [term 1 LEADER]: Becoming Leader. State: Replica: 079b49d95f494f1da866b1c948053a20, State: Running, Role: LEADER
I20260924 00:56:29.747795 13923 consensus_queue.cc:237] T 00000000000000000000000000000000 P 079b49d95f494f1da866b1c948053a20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "079b49d95f494f1da866b1c948053a20" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 34687 } }
I20260924 00:56:29.749228 13918 sys_catalog.cc:564] T 00000000000000000000000000000000 P 079b49d95f494f1da866b1c948053a20 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:56:29.751116 13925 sys_catalog.cc:455] T 00000000000000000000000000000000 P 079b49d95f494f1da866b1c948053a20 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "079b49d95f494f1da866b1c948053a20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "079b49d95f494f1da866b1c948053a20" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 34687 } } }
I20260924 00:56:29.751734 13925 sys_catalog.cc:458] T 00000000000000000000000000000000 P 079b49d95f494f1da866b1c948053a20 [sys.catalog]: This master's current role is: LEADER
I20260924 00:56:29.751788 13924 sys_catalog.cc:455] T 00000000000000000000000000000000 P 079b49d95f494f1da866b1c948053a20 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 079b49d95f494f1da866b1c948053a20. Latest consensus state: current_term: 1 leader_uuid: "079b49d95f494f1da866b1c948053a20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "079b49d95f494f1da866b1c948053a20" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 34687 } } }
I20260924 00:56:29.752393 13924 sys_catalog.cc:458] T 00000000000000000000000000000000 P 079b49d95f494f1da866b1c948053a20 [sys.catalog]: This master's current role is: LEADER
I20260924 00:56:29.757097 13930 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:56:29.762971 13930 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:56:29.774782 13930 catalog_manager.cc:1347] Generated new cluster ID: ea1cf136438747ba975e8430b0948345
I20260924 00:56:29.774971 13930 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:56:29.785779 13930 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:56:29.787312 13930 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:56:29.802719 13930 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 079b49d95f494f1da866b1c948053a20: Generated new TSK 0
I20260924 00:56:29.803855 13930 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 00:56:29.829176 13921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:29.820270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43512 (local address 127.8.64.190:34687)
0924 00:56:29.820464 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:29.820471 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:29.820789 (+ 318us) server_negotiation.cc:407] Connection header received
0924 00:56:29.821032 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:29.821038 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:29.821148 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:29.821343 (+ 195us) server_negotiation.cc:226] Negotiated authn=JWT
0924 00:56:29.821794 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:29.823091 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:29.824473 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:29.825221 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:29.825352 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:29.825869 (+ 517us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0924 00:56:29.828467 (+ 2598us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45499/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0924 00:56:29.828825 (+ 358us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43512: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45499/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":74}
W20260924 00:56:29.829311 13921 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43512: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45499/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260924 00:56:29.830835 13948 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:56:29.820080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:34687 (local address 127.0.0.1:43512)
0924 00:56:29.820681 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:29.820696 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:29.820828 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:29.821367 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:29.821370 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:29.821379 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0924 00:56:29.821637 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:29.821645 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:29.823265 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:29.823269 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:29.824327 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:29.824338 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:29.825416 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:29.825420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:29.825545 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:29.825576 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0924 00:56:29.828696 (+ 3120us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45499/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0924 00:56:29.830435 (+ 1739us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.64.190:34687: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45499/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":470,"mutex_wait_us":1411,"thread_start_us":345,"threads_started":1}
I20260924 00:56:29.831984 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 13852
I20260924 00:56:29.833355 13913 generic_service.cc:196] Checking for leaks (request via RPC)
E20260924 00:56:29.841317 13949 webserver.cc:533] Webserver: sslize failed:
E20260924 00:56:29.841445 13949 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260924 00:56:29.926149 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 13852
2026-09-24T00:56:29Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (716 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-24T00:56:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:56:29Z Disabled control of system clock
I20260924 00:56:30.008612 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:33965
--webserver_interface=127.8.64.190
--webserver_port=0
--builtin_ntp_servers=127.8.64.148:39781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:33965
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790211343099518-8450-0/testchainca.cert with env {}
W20260924 00:56:30.333158 13954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:30.333455 13954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:30.333603 13954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:30.341063 13954 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:56:30.341180 13954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:30.341254 13954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:30.341352 13954 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:56:30.341414 13954 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260924 00:56:30.341497 13954 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260924 00:56:30.341552 13954 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260924 00:56:30.352030 13954 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:39781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:33965
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:33965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790211343099518-8450-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:30.353363 13954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:30.354700 13954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:30.366370 13960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:30.366320 13959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:30.366802 13962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:30.367164 13954 server_base.cc:1034] running on GCE node
I20260924 00:56:30.367950 13954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:30.369307 13954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:30.370543 13954 hybrid_clock.cc:648] HybridClock initialized: now 1790211390370506 us; error 52 us; skew 500 ppm
I20260924 00:56:30.370915 13954 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 00:56:30.373220 13954 webserver.cc:460] Webserver started at http://127.8.64.190:38253/ using document root <none> and password file <none>
I20260924 00:56:30.373941 13954 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:30.374085 13954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:30.374465 13954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:56:30.377518 13954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "3e2d06d3465c44508b224f9413125c61"
format_stamp: "Formatted at 2026-09-24 00:56:30 on dist-test-slave-mmhs"
server_key: "9bd7109f3f7dbdfd07b2bf2ab642b6d6"
server_key_iv: "eec7743a8ee8c6c2c8342854c695aa08"
server_key_version: "encryptionkey@0"
I20260924 00:56:30.378419 13954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "3e2d06d3465c44508b224f9413125c61"
format_stamp: "Formatted at 2026-09-24 00:56:30 on dist-test-slave-mmhs"
server_key: "9bd7109f3f7dbdfd07b2bf2ab642b6d6"
server_key_iv: "eec7743a8ee8c6c2c8342854c695aa08"
server_key_version: "encryptionkey@0"
I20260924 00:56:30.383752 13954 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260924 00:56:30.387435 13968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:30.388974 13954 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260924 00:56:30.389170 13954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "3e2d06d3465c44508b224f9413125c61"
format_stamp: "Formatted at 2026-09-24 00:56:30 on dist-test-slave-mmhs"
server_key: "9bd7109f3f7dbdfd07b2bf2ab642b6d6"
server_key_iv: "eec7743a8ee8c6c2c8342854c695aa08"
server_key_version: "encryptionkey@0"
I20260924 00:56:30.389423 13954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:30.413230 13954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:30.414152 13954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:30.414505 13954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:30.433377 13954 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.190:33965
I20260924 00:56:30.433394 14019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.190:33965 every 8 connection(s)
I20260924 00:56:30.434806 13954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 00:56:30.438036 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 13954
I20260924 00:56:30.438227 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 00:56:30.439043 14020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:30.439059 8450 external_mini_cluster.cc:1428] Setting key b1fd3ab5155797d72d9895009c689cfc
I20260924 00:56:30.452140 14020 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e2d06d3465c44508b224f9413125c61: Bootstrap starting.
I20260924 00:56:30.452869 14023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:30.440976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54684 (local address 127.8.64.190:33965)
0924 00:56:30.441913 (+ 937us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:30.441924 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:30.441967 (+ 43us) server_negotiation.cc:407] Connection header received
0924 00:56:30.443822 (+ 1855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:30.443843 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:30.444332 (+ 489us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:30.444735 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:30.445326 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:30.446657 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:30.447916 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:30.448605 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:30.448829 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:30.450086 (+ 1257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:30.450096 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:30.450115 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:56:30.450846 (+ 731us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:30.450940 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:56:30.450966 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:30.451083 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:30.451299 (+ 216us) server_negotiation.cc:299] Negotiation successful
0924 00:56:30.451613 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514,"thread_start_us":317,"threads_started":1}
I20260924 00:56:30.455862 14020 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e2d06d3465c44508b224f9413125c61: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:30.456645 8450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 00:56:30.457041 14020 log.cc:824] T 00000000000000000000000000000000 P 3e2d06d3465c44508b224f9413125c61: Log is configured to *not* fsync() on all Append() calls
I20260924 00:56:30.460990 14020 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e2d06d3465c44508b224f9413125c61: No bootstrap required, opened a new log
I20260924 00:56:30.468071 14020 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e2d06d3465c44508b224f9413125c61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e2d06d3465c44508b224f9413125c61" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 33965 } }
I20260924 00:56:30.468468 14020 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e2d06d3465c44508b224f9413125c61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:30.468576 14020 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e2d06d3465c44508b224f9413125c61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e2d06d3465c44508b224f9413125c61, State: Initialized, Role: FOLLOWER
I20260924 00:56:30.469137 14020 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e2d06d3465c44508b224f9413125c61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e2d06d3465c44508b224f9413125c61" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 33965 } }
I20260924 00:56:30.469385 14020 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e2d06d3465c44508b224f9413125c61 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:56:30.469552 14020 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e2d06d3465c44508b224f9413125c61 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:56:30.469758 14020 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e2d06d3465c44508b224f9413125c61 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:30.471964 14020 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e2d06d3465c44508b224f9413125c61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e2d06d3465c44508b224f9413125c61" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 33965 } }
I20260924 00:56:30.472401 14020 leader_election.cc:302] T 00000000000000000000000000000000 P 3e2d06d3465c44508b224f9413125c61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e2d06d3465c44508b224f9413125c61; no voters:
I20260924 00:56:30.472937 14020 leader_election.cc:288] T 00000000000000000000000000000000 P 3e2d06d3465c44508b224f9413125c61 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:56:30.473138 14025 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e2d06d3465c44508b224f9413125c61 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:56:30.473891 14025 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e2d06d3465c44508b224f9413125c61 [term 1 LEADER]: Becoming Leader. State: Replica: 3e2d06d3465c44508b224f9413125c61, State: Running, Role: LEADER
I20260924 00:56:30.474493 14025 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e2d06d3465c44508b224f9413125c61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e2d06d3465c44508b224f9413125c61" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 33965 } }
I20260924 00:56:30.475128 14020 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e2d06d3465c44508b224f9413125c61 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:56:30.477363 14027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e2d06d3465c44508b224f9413125c61 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e2d06d3465c44508b224f9413125c61. Latest consensus state: current_term: 1 leader_uuid: "3e2d06d3465c44508b224f9413125c61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e2d06d3465c44508b224f9413125c61" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 33965 } } }
I20260924 00:56:30.477715 14027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e2d06d3465c44508b224f9413125c61 [sys.catalog]: This master's current role is: LEADER
I20260924 00:56:30.481690 14026 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e2d06d3465c44508b224f9413125c61 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3e2d06d3465c44508b224f9413125c61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e2d06d3465c44508b224f9413125c61" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 33965 } } }
I20260924 00:56:30.482360 14026 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e2d06d3465c44508b224f9413125c61 [sys.catalog]: This master's current role is: LEADER
I20260924 00:56:30.483767 14031 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:56:30.489078 14031 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:56:30.498029 14031 catalog_manager.cc:1347] Generated new cluster ID: c08465aeb19b420f991d36068449220a
I20260924 00:56:30.498178 14031 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:56:30.553234 14031 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:56:30.554625 14031 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:56:30.561973 14031 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3e2d06d3465c44508b224f9413125c61: Generated new TSK 0
I20260924 00:56:30.562911 14031 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 00:56:31.050897 14023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:30.576335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54686 (local address 127.8.64.190:33965)
0924 00:56:30.576524 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:30.576529 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:30.576899 (+ 370us) server_negotiation.cc:407] Connection header received
0924 00:56:30.577119 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:30.577126 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:30.577236 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:30.577413 (+ 177us) server_negotiation.cc:226] Negotiated authn=JWT
0924 00:56:30.577844 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:30.579047 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:30.580402 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:30.581134 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:30.581255 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:30.581756 (+ 501us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0924 00:56:31.050342 (+468586us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0924 00:56:31.050687 (+ 345us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54686: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":43}
W20260924 00:56:31.051079 14023 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54686: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260924 00:56:31.051270 14049 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:56:30.576162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:33965 (local address 127.0.0.1:54686)
0924 00:56:30.576797 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:30.576813 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:30.576933 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:30.577427 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:30.577430 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:30.577440 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0924 00:56:30.577704 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:30.577711 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:30.579193 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:30.579197 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:30.580256 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:30.580266 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:30.581318 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:30.581323 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:30.581449 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:30.581481 (+ 32us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0924 00:56:31.050617 (+469136us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0924 00:56:31.050872 (+ 255us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.64.190:33965: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":349,"threads_started":1}
I20260924 00:56:31.052209 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 13954
I20260924 00:56:31.053426 14015 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:56:31.154506 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 13954
2026-09-24T00:56:31Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1209 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (409 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (412 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:56:32 dist-test-slave-mmhs krb5kdc[14064](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:56:32 dist-test-slave-mmhs krb5kdc[14064](info): set up 2 sockets
Sep 24 00:56:32 dist-test-slave-mmhs krb5kdc[14064](info): commencing operation
krb5kdc: starting...
W20260924 00:56:33.073055 8450 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Sep 24 00:56:33 dist-test-slave-mmhs krb5kdc[14064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211393, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:56:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:56:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.64.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
I20260924 00:56:33.287262 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:37751
--webserver_interface=127.8.64.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:35317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:37751
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:56:33.606324 14077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:33.606597 14077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:33.606809 14077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:33.614434 14077 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:56:33.614563 14077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:33.614646 14077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:33.614730 14077 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:56:33.614813 14077 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:56:33.624341 14077 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:35317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:37751
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:37751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:33.625610 14077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:33.626950 14077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:33.638520 14082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:33.638603 14083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:33.638557 14085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:33.639789 14077 server_base.cc:1034] running on GCE node
I20260924 00:56:33.640532 14077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:33.641922 14077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:33.643121 14077 hybrid_clock.cc:648] HybridClock initialized: now 1790211393643078 us; error 29 us; skew 500 ppm
I20260924 00:56:33.643468 14077 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:33 dist-test-slave-mmhs krb5kdc[14064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211393, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:33.646512 14077 init.cc:377] Logged in from keytab as kudu/127.8.64.190@KRBTEST.COM (short username kudu)
I20260924 00:56:33.648023 14077 webserver.cc:460] Webserver started at http://127.8.64.190:34539/ using document root <none> and password file <none>
I20260924 00:56:33.648695 14077 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:33.648840 14077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:33.649150 14077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:56:33.652099 14077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "ca8c095bee404173b83ca88b780b2e38"
format_stamp: "Formatted at 2026-09-24 00:56:33 on dist-test-slave-mmhs"
server_key: "d927d95edef26d518803c18c94ab18bb"
server_key_iv: "c286eef51319eabd218d4bd5d2830aad"
server_key_version: "encryptionkey@0"
I20260924 00:56:33.652885 14077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "ca8c095bee404173b83ca88b780b2e38"
format_stamp: "Formatted at 2026-09-24 00:56:33 on dist-test-slave-mmhs"
server_key: "d927d95edef26d518803c18c94ab18bb"
server_key_iv: "c286eef51319eabd218d4bd5d2830aad"
server_key_version: "encryptionkey@0"
I20260924 00:56:33.658124 14077 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260924 00:56:33.661499 14092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:33.662814 14077 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 00:56:33.663014 14077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "ca8c095bee404173b83ca88b780b2e38"
format_stamp: "Formatted at 2026-09-24 00:56:33 on dist-test-slave-mmhs"
server_key: "d927d95edef26d518803c18c94ab18bb"
server_key_iv: "c286eef51319eabd218d4bd5d2830aad"
server_key_version: "encryptionkey@0"
I20260924 00:56:33.663219 14077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:33.680213 14077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:33.684198 14077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:33.684502 14077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:33.703282 14077 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.190:37751
I20260924 00:56:33.703328 14143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.190:37751 every 8 connection(s)
I20260924 00:56:33.704682 14077 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 00:56:33.707564 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 14077
I20260924 00:56:33.707777 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 00:56:33.708384 8450 external_mini_cluster.cc:1428] Setting key f30df374f4d8477ba229eba6be813291
I20260924 00:56:33.708621 14144 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:33.723026 14144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca8c095bee404173b83ca88b780b2e38: Bootstrap starting.
Sep 24 00:56:33 dist-test-slave-mmhs krb5kdc[14064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211393, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:56:33.726513 14144 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca8c095bee404173b83ca88b780b2e38: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:33.727934 14144 log.cc:824] T 00000000000000000000000000000000 P ca8c095bee404173b83ca88b780b2e38: Log is configured to *not* fsync() on all Append() calls
I20260924 00:56:33.731344 14144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca8c095bee404173b83ca88b780b2e38: No bootstrap required, opened a new log
I20260924 00:56:33.733202 14147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:33.710390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38370 (local address 127.8.64.190:37751)
0924 00:56:33.711154 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:33.711164 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:33.711205 (+ 41us) server_negotiation.cc:407] Connection header received
0924 00:56:33.713429 (+ 2224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:33.713452 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:33.713885 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:33.714240 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:33.716132 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:33.717442 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:33.719346 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:33.720068 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:33.720225 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:33.724914 (+ 4689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:33.724922 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:33.724936 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:33.724976 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:33.728720 (+ 3744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:33.729551 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:33.729559 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:33.729568 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:33.729765 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:33.730123 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:33.730131 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:33.730135 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:33.730879 (+ 744us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:33.731172 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:33.731470 (+ 298us) server_negotiation.cc:299] Negotiation successful
0924 00:56:33.731917 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":348,"threads_started":1}
I20260924 00:56:33.739887 14144 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca8c095bee404173b83ca88b780b2e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca8c095bee404173b83ca88b780b2e38" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 37751 } }
I20260924 00:56:33.740213 14144 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca8c095bee404173b83ca88b780b2e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:33.740291 14144 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca8c095bee404173b83ca88b780b2e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca8c095bee404173b83ca88b780b2e38, State: Initialized, Role: FOLLOWER
I20260924 00:56:33.740824 14144 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca8c095bee404173b83ca88b780b2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca8c095bee404173b83ca88b780b2e38" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 37751 } }
I20260924 00:56:33.741014 14144 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca8c095bee404173b83ca88b780b2e38 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:56:33.741104 14144 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca8c095bee404173b83ca88b780b2e38 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:56:33.741218 14144 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca8c095bee404173b83ca88b780b2e38 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:33.743310 14144 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca8c095bee404173b83ca88b780b2e38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca8c095bee404173b83ca88b780b2e38" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 37751 } }
I20260924 00:56:33.743729 14144 leader_election.cc:302] T 00000000000000000000000000000000 P ca8c095bee404173b83ca88b780b2e38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca8c095bee404173b83ca88b780b2e38; no voters:
I20260924 00:56:33.744248 14144 leader_election.cc:288] T 00000000000000000000000000000000 P ca8c095bee404173b83ca88b780b2e38 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:56:33.744769 14149 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca8c095bee404173b83ca88b780b2e38 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:56:33.745832 14149 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca8c095bee404173b83ca88b780b2e38 [term 1 LEADER]: Becoming Leader. State: Replica: ca8c095bee404173b83ca88b780b2e38, State: Running, Role: LEADER
I20260924 00:56:33.746392 14149 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca8c095bee404173b83ca88b780b2e38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca8c095bee404173b83ca88b780b2e38" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 37751 } }
I20260924 00:56:33.746739 14144 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca8c095bee404173b83ca88b780b2e38 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:56:33.749313 14151 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca8c095bee404173b83ca88b780b2e38 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca8c095bee404173b83ca88b780b2e38. Latest consensus state: current_term: 1 leader_uuid: "ca8c095bee404173b83ca88b780b2e38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca8c095bee404173b83ca88b780b2e38" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 37751 } } }
I20260924 00:56:33.749733 14151 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca8c095bee404173b83ca88b780b2e38 [sys.catalog]: This master's current role is: LEADER
I20260924 00:56:33.749289 14150 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca8c095bee404173b83ca88b780b2e38 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca8c095bee404173b83ca88b780b2e38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca8c095bee404173b83ca88b780b2e38" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 37751 } } }
I20260924 00:56:33.750278 14150 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca8c095bee404173b83ca88b780b2e38 [sys.catalog]: This master's current role is: LEADER
I20260924 00:56:33.750623 14157 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:56:33.755633 14157 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:56:33.769676 14157 catalog_manager.cc:1347] Generated new cluster ID: 078f3b42ace648e685b8396db28cbb30
I20260924 00:56:33.769805 14157 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:56:33.808283 14157 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:56:33.809608 14157 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:56:33.816672 14157 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca8c095bee404173b83ca88b780b2e38: Generated new TSK 0
I20260924 00:56:33.817474 14157 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.64.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab.
Entry for principal kudu/127.8.64.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab.
I20260924 00:56:33.878459 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.129:0
--local_ip_for_outbound_sockets=127.8.64.129
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:37751
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab
--principal=kudu/127.8.64.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:35317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:56:34.204934 14170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:34.205222 14170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:34.205379 14170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:34.212473 14170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:34.212579 14170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:34.212755 14170 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.129
I20260924 00:56:34.222167 14170 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:35317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab
--principal=kudu/127.8.64.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:37751
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.129
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:34.223444 14170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:34.224715 14170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:34.238112 14175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:34.238144 14178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:34.238269 14176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:34.238867 14170 server_base.cc:1034] running on GCE node
I20260924 00:56:34.239463 14170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:34.240440 14170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:34.241677 14170 hybrid_clock.cc:648] HybridClock initialized: now 1790211394241618 us; error 69 us; skew 500 ppm
I20260924 00:56:34.241998 14170 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:34 dist-test-slave-mmhs krb5kdc[14064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211394, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:34.249579 14170 init.cc:377] Logged in from keytab as kudu/127.8.64.129@KRBTEST.COM (short username kudu)
I20260924 00:56:34.251096 14170 webserver.cc:460] Webserver started at http://127.8.64.129:40379/ using document root <none> and password file <none>
I20260924 00:56:34.251780 14170 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:34.251936 14170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:34.252249 14170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:56:34.255236 14170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-0/data/instance:
uuid: "ce863a4a4ee64c5cb4a7ecc416c99f7f"
format_stamp: "Formatted at 2026-09-24 00:56:34 on dist-test-slave-mmhs"
server_key: "e17e19992b441c0bdc649a78847eeb20"
server_key_iv: "eb339a895b60bed2934729adfa7e8f3b"
server_key_version: "encryptionkey@0"
I20260924 00:56:34.256031 14170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance:
uuid: "ce863a4a4ee64c5cb4a7ecc416c99f7f"
format_stamp: "Formatted at 2026-09-24 00:56:34 on dist-test-slave-mmhs"
server_key: "e17e19992b441c0bdc649a78847eeb20"
server_key_iv: "eb339a895b60bed2934729adfa7e8f3b"
server_key_version: "encryptionkey@0"
I20260924 00:56:34.261240 14170 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260924 00:56:34.264775 14185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:34.266078 14170 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260924 00:56:34.266275 14170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-0/wal
uuid: "ce863a4a4ee64c5cb4a7ecc416c99f7f"
format_stamp: "Formatted at 2026-09-24 00:56:34 on dist-test-slave-mmhs"
server_key: "e17e19992b441c0bdc649a78847eeb20"
server_key_iv: "eb339a895b60bed2934729adfa7e8f3b"
server_key_version: "encryptionkey@0"
I20260924 00:56:34.266541 14170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:34.283092 14170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:34.287194 14170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:34.287499 14170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:34.288457 14170 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:56:34.289829 14170 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:56:34.289948 14170 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:34.290100 14170 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:56:34.290174 14170 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:34.323529 14170 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.129:40395
I20260924 00:56:34.323568 14297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.129:40395 every 8 connection(s)
I20260924 00:56:34.324836 14170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
I20260924 00:56:34.331862 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 14170
I20260924 00:56:34.332074 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance
I20260924 00:56:34.332697 8450 external_mini_cluster.cc:1428] Setting key cb5433b3016e3621f64eb052ae54c10a
Sep 24 00:56:34 dist-test-slave-mmhs krb5kdc[14064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211394, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:56:34.345112 14300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:34.327297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:58629 (local address 127.8.64.190:37751)
0924 00:56:34.327726 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:34.327730 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:34.328548 (+ 818us) server_negotiation.cc:407] Connection header received
0924 00:56:34.330343 (+ 1795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:34.330349 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:34.330442 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:34.330557 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:34.332256 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:34.333050 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:34.335015 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:34.335409 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:34.335507 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:34.339638 (+ 4131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:34.339643 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:34.339647 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:34.339677 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:34.342182 (+ 2505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:34.342913 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:34.342920 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:34.342924 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:34.343043 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:34.343472 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:34.343477 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:34.343480 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:34.343734 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:34.343896 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:34.344550 (+ 654us) server_negotiation.cc:299] Negotiation successful
0924 00:56:34.344758 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":238,"threads_started":1}
I20260924 00:56:34.346063 14301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:34.327696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:37751 (local address 127.8.64.129:58629)
0924 00:56:34.328390 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:34.328428 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:34.330041 (+ 1613us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:34.330791 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:34.330799 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:34.331276 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:34.332077 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:34.332104 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:34.333217 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:34.333224 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:34.334858 (+ 1634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:34.334873 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:34.335633 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:34.335642 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:34.335996 (+ 354us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:34.336725 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:34.336765 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:34.339410 (+ 2645us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:34.342345 (+ 2935us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:34.342355 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:34.342372 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:34.342785 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:34.343154 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:34.343160 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:34.343164 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:34.343352 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:34.343865 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:34.343874 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:34.344191 (+ 317us) client_negotiation.cc:769] Sending connection context
0924 00:56:34.344452 (+ 261us) client_negotiation.cc:240] Negotiation successful
0924 00:56:34.344964 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":287,"threads_started":1}
I20260924 00:56:34.347743 14298 heartbeater.cc:347] Connected to a master server at 127.8.64.190:37751
I20260924 00:56:34.348061 14298 heartbeater.cc:464] Registering TS with master...
I20260924 00:56:34.348762 14298 heartbeater.cc:511] Master 127.8.64.190:37751 requested a full tablet report, sending...
I20260924 00:56:34.351105 14109 ts_manager.cc:194] Registered new tserver with Master: ce863a4a4ee64c5cb4a7ecc416c99f7f (127.8.64.129:40395)
WARNING: no policy specified for kudu/127.8.64.130@KRBTEST.COM; defaulting to no policy
I20260924 00:56:34.353259 14109 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.129@KRBTEST.COM'} at 127.8.64.129:58629
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab.
Entry for principal kudu/127.8.64.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab.
I20260924 00:56:34.382480 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.130:0
--local_ip_for_outbound_sockets=127.8.64.130
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:37751
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab
--principal=kudu/127.8.64.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:35317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:56:34.705726 14304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:34.705983 14304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:34.706149 14304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:34.713364 14304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:34.713466 14304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:34.713650 14304 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.130
I20260924 00:56:34.723340 14304 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:35317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab
--principal=kudu/127.8.64.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:37751
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.130
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:34.724628 14304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:34.725926 14304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:34.739595 14312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:34.739714 14309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:34.739733 14310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:34.740401 14304 server_base.cc:1034] running on GCE node
I20260924 00:56:34.741089 14304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:34.742023 14304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:34.743211 14304 hybrid_clock.cc:648] HybridClock initialized: now 1790211394743144 us; error 71 us; skew 500 ppm
I20260924 00:56:34.743487 14304 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:34 dist-test-slave-mmhs krb5kdc[14064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211394, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:34.746459 14304 init.cc:377] Logged in from keytab as kudu/127.8.64.130@KRBTEST.COM (short username kudu)
I20260924 00:56:34.748039 14304 webserver.cc:460] Webserver started at http://127.8.64.130:42277/ using document root <none> and password file <none>
I20260924 00:56:34.748734 14304 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:34.748885 14304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:34.749194 14304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:56:34.752315 14304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-1/data/instance:
uuid: "2c04586a8e304da6bb53c8b00d8f5f14"
format_stamp: "Formatted at 2026-09-24 00:56:34 on dist-test-slave-mmhs"
server_key: "c0b3901b8bc3682f0a165289096dcfa4"
server_key_iv: "42f9853a97076eb0355e12530486cf00"
server_key_version: "encryptionkey@0"
I20260924 00:56:34.753098 14304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance:
uuid: "2c04586a8e304da6bb53c8b00d8f5f14"
format_stamp: "Formatted at 2026-09-24 00:56:34 on dist-test-slave-mmhs"
server_key: "c0b3901b8bc3682f0a165289096dcfa4"
server_key_iv: "42f9853a97076eb0355e12530486cf00"
server_key_version: "encryptionkey@0"
I20260924 00:56:34.758361 14304 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260924 00:56:34.761869 14319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:34.763334 14304 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 00:56:34.763499 14304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-1/wal
uuid: "2c04586a8e304da6bb53c8b00d8f5f14"
format_stamp: "Formatted at 2026-09-24 00:56:34 on dist-test-slave-mmhs"
server_key: "c0b3901b8bc3682f0a165289096dcfa4"
server_key_iv: "42f9853a97076eb0355e12530486cf00"
server_key_version: "encryptionkey@0"
I20260924 00:56:34.763706 14304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:34.780490 14304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:34.784302 14304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:34.784588 14304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:34.785506 14304 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:56:34.786855 14304 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:56:34.786957 14304 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:34.787102 14304 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:56:34.787173 14304 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:34.820251 14304 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.130:33165
I20260924 00:56:34.820297 14431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.130:33165 every 8 connection(s)
I20260924 00:56:34.821548 14304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
I20260924 00:56:34.822544 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 14304
I20260924 00:56:34.822700 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance
I20260924 00:56:34.823226 8450 external_mini_cluster.cc:1428] Setting key ea99ba31a1e94205203c78a32347e58e
Sep 24 00:56:34 dist-test-slave-mmhs krb5kdc[14064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211394, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.130@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:56:34.843113 14300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:34.824430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:39753 (local address 127.8.64.190:37751)
0924 00:56:34.824628 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:34.824634 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:34.826089 (+ 1455us) server_negotiation.cc:407] Connection header received
0924 00:56:34.827845 (+ 1756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:34.827853 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:34.827945 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:34.828047 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:34.829921 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:34.831052 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:34.832528 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:34.832947 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:34.833086 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:34.836432 (+ 3346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:34.836441 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:34.836448 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:34.836492 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:34.840041 (+ 3549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:34.840781 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:34.840788 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:34.840795 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:34.840929 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:34.841366 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:34.841373 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:34.841376 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:34.841629 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:34.841797 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:34.842545 (+ 748us) server_negotiation.cc:299] Negotiation successful
0924 00:56:34.842775 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260924 00:56:34.844064 14434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:34.825054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:37751 (local address 127.8.64.130:39753)
0924 00:56:34.825927 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:34.825973 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:34.827591 (+ 1618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:34.828218 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:34.828251 (+ 33us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:34.829000 (+ 749us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:34.829744 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:34.829770 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:34.831201 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:34.831205 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:34.832380 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:34.832392 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:34.833169 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:34.833175 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:34.833411 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:34.834074 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:34.834099 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:34.836213 (+ 2114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:34.840195 (+ 3982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:34.840202 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:34.840215 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:34.840632 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:34.841040 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:34.841046 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:34.841050 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:34.841226 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:34.841763 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:34.841770 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:34.842103 (+ 333us) client_negotiation.cc:769] Sending connection context
0924 00:56:34.842445 (+ 342us) client_negotiation.cc:240] Negotiation successful
0924 00:56:34.842942 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":387,"threads_started":1}
WARNING: no policy specified for kudu/127.8.64.131@KRBTEST.COM; defaulting to no policy
I20260924 00:56:34.845458 14432 heartbeater.cc:347] Connected to a master server at 127.8.64.190:37751
I20260924 00:56:34.845763 14432 heartbeater.cc:464] Registering TS with master...
I20260924 00:56:34.846565 14432 heartbeater.cc:511] Master 127.8.64.190:37751 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.131@KRBTEST.COM" created.
I20260924 00:56:34.848129 14109 ts_manager.cc:194] Registered new tserver with Master: 2c04586a8e304da6bb53c8b00d8f5f14 (127.8.64.130:33165)
I20260924 00:56:34.849380 14109 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.130@KRBTEST.COM'} at 127.8.64.130:39753
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab.
Entry for principal kudu/127.8.64.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab.
I20260924 00:56:34.874109 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.131:0
--local_ip_for_outbound_sockets=127.8.64.131
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:37751
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab
--principal=kudu/127.8.64.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:35317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:56:35.210400 14437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:35.210675 14437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:35.210767 14437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:35.217782 14437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:35.217883 14437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:35.217994 14437 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.131
I20260924 00:56:35.227506 14437 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:35317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab
--principal=kudu/127.8.64.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:37751
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.131
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:35.228693 14437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:35.229923 14437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:35.243424 14442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:35.243391 14445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:35.243458 14443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:35.244153 14437 server_base.cc:1034] running on GCE node
I20260924 00:56:35.244753 14437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:35.245606 14437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:35.246836 14437 hybrid_clock.cc:648] HybridClock initialized: now 1790211395246789 us; error 57 us; skew 500 ppm
I20260924 00:56:35.247187 14437 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:35 dist-test-slave-mmhs krb5kdc[14064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211395, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:35.250442 14437 init.cc:377] Logged in from keytab as kudu/127.8.64.131@KRBTEST.COM (short username kudu)
I20260924 00:56:35.251909 14437 webserver.cc:460] Webserver started at http://127.8.64.131:33963/ using document root <none> and password file <none>
I20260924 00:56:35.252588 14437 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:35.252736 14437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:35.253083 14437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:56:35.256062 14437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-2/data/instance:
uuid: "59716394f705469683fbe1c06d7b78b0"
format_stamp: "Formatted at 2026-09-24 00:56:35 on dist-test-slave-mmhs"
server_key: "f739cd9661279682f610252e00da4741"
server_key_iv: "38f0129762c9c1639b220d326681014a"
server_key_version: "encryptionkey@0"
I20260924 00:56:35.256855 14437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance:
uuid: "59716394f705469683fbe1c06d7b78b0"
format_stamp: "Formatted at 2026-09-24 00:56:35 on dist-test-slave-mmhs"
server_key: "f739cd9661279682f610252e00da4741"
server_key_iv: "38f0129762c9c1639b220d326681014a"
server_key_version: "encryptionkey@0"
I20260924 00:56:35.261938 14437 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260924 00:56:35.265554 14452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:35.266903 14437 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260924 00:56:35.267073 14437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-2/wal
uuid: "59716394f705469683fbe1c06d7b78b0"
format_stamp: "Formatted at 2026-09-24 00:56:35 on dist-test-slave-mmhs"
server_key: "f739cd9661279682f610252e00da4741"
server_key_iv: "38f0129762c9c1639b220d326681014a"
server_key_version: "encryptionkey@0"
I20260924 00:56:35.267272 14437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:35.284757 14437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:35.288625 14437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:35.288919 14437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:35.289868 14437 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:56:35.291246 14437 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:56:35.291356 14437 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:35.291512 14437 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:56:35.291599 14437 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:35.323869 14437 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.131:37059
I20260924 00:56:35.323886 14564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.131:37059 every 8 connection(s)
I20260924 00:56:35.325445 14437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
I20260924 00:56:35.334511 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 14437
I20260924 00:56:35.334668 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance
I20260924 00:56:35.335088 8450 external_mini_cluster.cc:1428] Setting key dd13e7bc4b0dbca8dc3a0f042af06d6b
Sep 24 00:56:35 dist-test-slave-mmhs krb5kdc[14064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211395, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.131@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:56:35.345027 14300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:35.328169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:41409 (local address 127.8.64.190:37751)
0924 00:56:35.328392 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:35.328399 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:35.329567 (+ 1168us) server_negotiation.cc:407] Connection header received
0924 00:56:35.331331 (+ 1764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:35.331335 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:35.331425 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:35.331521 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:35.332852 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:35.333572 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:35.335331 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:35.335895 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:35.336015 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:35.339776 (+ 3761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:35.339781 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:35.339784 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:35.339816 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:35.342444 (+ 2628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:35.343066 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:35.343070 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:35.343072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:35.343189 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:35.343548 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:35.343551 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:35.343553 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:35.343793 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:35.343901 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:35.344538 (+ 637us) server_negotiation.cc:299] Negotiation successful
0924 00:56:35.344765 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260924 00:56:35.345871 14567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:35.328594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:37751 (local address 127.8.64.131:41409)
0924 00:56:35.329403 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:35.329445 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:35.331086 (+ 1641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:35.331720 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:35.331728 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:35.332183 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:35.332712 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:35.332728 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:35.333720 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:35.333725 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:35.335154 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:35.335201 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:35.336121 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:35.336130 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:35.336458 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:35.337209 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:35.337246 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:35.339594 (+ 2348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:35.342557 (+ 2963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:35.342563 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:35.342575 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:35.342965 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:35.343274 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:35.343278 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:35.343280 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:35.343436 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:35.343894 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:35.343899 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:35.344211 (+ 312us) client_negotiation.cc:769] Sending connection context
0924 00:56:35.344419 (+ 208us) client_negotiation.cc:240] Negotiation successful
0924 00:56:35.344791 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":347,"threads_started":1}
I20260924 00:56:35.347291 14565 heartbeater.cc:347] Connected to a master server at 127.8.64.190:37751
I20260924 00:56:35.347692 14565 heartbeater.cc:464] Registering TS with master...
I20260924 00:56:35.348470 14565 heartbeater.cc:511] Master 127.8.64.190:37751 requested a full tablet report, sending...
I20260924 00:56:35.350181 14109 ts_manager.cc:194] Registered new tserver with Master: 59716394f705469683fbe1c06d7b78b0 (127.8.64.131:37059)
I20260924 00:56:35.351018 14109 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.131@KRBTEST.COM'} at 127.8.64.131:41409
I20260924 00:56:35.356264 14298 heartbeater.cc:503] Master 127.8.64.190:37751 was elected leader, sending a full tablet report...
I20260924 00:56:35.362529 8450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/krb5kdc/test-admin.keytab.
Sep 24 00:56:35 dist-test-slave-mmhs krb5kdc[14064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:35.391585 8450 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260924 00:56:35.396678 14572 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:56:35.394259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54926 (local address 127.8.64.129:40395)
0924 00:56:35.394934 (+ 675us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:35.394941 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:35.394967 (+ 26us) server_negotiation.cc:407] Connection header received
0924 00:56:35.395030 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:35.395036 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:35.395242 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:35.395381 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:35.396280 (+ 899us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54926: BlockingRecv error: recv got EOF from 127.0.0.1:54926 (error 108)
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":350,"threads_started":1}
W20260924 00:56:35.397130 8450 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/krb5kdc/krb5cc)
Sep 24 00:56:35 dist-test-slave-mmhs krb5kdc[14064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:35.398749 8450 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 24 00:56:35 dist-test-slave-mmhs krb5kdc[14064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:35.400101 8450 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 24 00:56:35 dist-test-slave-mmhs krb5kdc[14064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.129@KRBTEST.COM
I20260924 00:56:35.415063 14572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:35.401597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54938 (local address 127.8.64.129:40395)
0924 00:56:35.401805 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:35.401812 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:35.402034 (+ 222us) server_negotiation.cc:407] Connection header received
0924 00:56:35.402215 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:35.402226 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:35.402330 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:35.402443 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:35.403405 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:35.404349 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:35.405527 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:35.406074 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:35.406216 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:35.409570 (+ 3354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:35.409580 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:35.409593 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:35.409621 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:35.412710 (+ 3089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:35.413303 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:35.413309 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:35.413312 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:35.413415 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:35.413714 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:35.413717 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:35.413719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:35.414039 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:35.414249 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:35.414611 (+ 362us) server_negotiation.cc:299] Negotiation successful
0924 00:56:35.414809 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260924 00:56:35.420814 14572 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:56:35.419028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54952 (local address 127.8.64.129:40395)
0924 00:56:35.419236 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:35.419241 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:35.419482 (+ 241us) server_negotiation.cc:407] Connection header received
0924 00:56:35.419678 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:35.419682 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:35.419747 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:35.419845 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:35.420547 (+ 702us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54952: BlockingRecv error: recv got EOF from 127.0.0.1:54952 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260924 00:56:35.421006 8450 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/krb5kdc/krb5cc' of type FILE
Sep 24 00:56:35 dist-test-slave-mmhs krb5kdc[14064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:35.422513 8450 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 24 00:56:35 dist-test-slave-mmhs krb5kdc[14064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:35.423789 8450 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 24 00:56:35 dist-test-slave-mmhs krb5kdc[14064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.129@KRBTEST.COM
I20260924 00:56:35.437333 14572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:35.424973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54954 (local address 127.8.64.129:40395)
0924 00:56:35.425180 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:35.425188 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:35.425455 (+ 267us) server_negotiation.cc:407] Connection header received
0924 00:56:35.425635 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:35.425641 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:35.425692 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:35.425792 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:35.426735 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:35.427424 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:35.428627 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:35.429053 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:35.429189 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:35.432502 (+ 3313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:35.432507 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:35.432510 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:35.432539 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:35.434951 (+ 2412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:35.435540 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:35.435546 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:35.435550 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:35.435673 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:35.436080 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:35.436086 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:35.436090 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:35.436343 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:35.436605 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:35.436864 (+ 259us) server_negotiation.cc:299] Negotiation successful
0924 00:56:35.437057 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260924 00:56:35.441258 14572 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:56:35.439643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54962 (local address 127.8.64.129:40395)
0924 00:56:35.439820 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:35.439826 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:35.440188 (+ 362us) server_negotiation.cc:407] Connection header received
0924 00:56:35.440308 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:35.440312 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:35.440369 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:35.440446 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:35.441021 (+ 575us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54962: BlockingRecv error: recv got EOF from 127.0.0.1:54962 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260924 00:56:35.441480 8450 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790211343099518-8450-0/krb5kdc/krb5cc' of type FILE
Sep 24 00:56:35 dist-test-slave-mmhs krb5kdc[14064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:35.442974 8450 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 24 00:56:35 dist-test-slave-mmhs krb5kdc[14064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:35.444176 8450 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 24 00:56:35 dist-test-slave-mmhs krb5kdc[14064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.129@KRBTEST.COM
I20260924 00:56:35.457670 14572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:35.445338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54966 (local address 127.8.64.129:40395)
0924 00:56:35.445546 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:35.445551 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:35.445763 (+ 212us) server_negotiation.cc:407] Connection header received
0924 00:56:35.445880 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:35.445884 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:35.445934 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:35.446008 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:35.447004 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:35.447684 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:35.449025 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:35.449433 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:35.449560 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:35.452875 (+ 3315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:35.452881 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:35.452887 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:35.452919 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:35.455227 (+ 2308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:35.455850 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:35.455856 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:35.455860 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:35.455990 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:35.456355 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:35.456362 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:35.456365 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:35.456616 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:35.456926 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:35.457159 (+ 233us) server_negotiation.cc:299] Negotiation successful
0924 00:56:35.457356 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260924 00:56:35.459071 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 14170
I20260924 00:56:35.470376 14572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:35.459753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54974 (local address 127.8.64.129:40395)
0924 00:56:35.459889 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:35.459893 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:35.460312 (+ 419us) server_negotiation.cc:407] Connection header received
0924 00:56:35.460387 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:35.460391 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:35.460443 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:35.460524 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:35.461405 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:35.462100 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:35.463389 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:35.463778 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:35.463905 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:35.465654 (+ 1749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:35.465658 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:35.465661 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:35.465687 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:35.468208 (+ 2521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:35.468792 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:35.468795 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:35.468798 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:35.468911 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:35.469200 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:35.469203 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:35.469204 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:35.469450 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:35.469655 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:35.469921 (+ 266us) server_negotiation.cc:299] Negotiation successful
0924 00:56:35.470065 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260924 00:56:35.471184 14293 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:56:35.553719 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 14170
I20260924 00:56:35.570902 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 14304
Sep 24 00:56:35 dist-test-slave-mmhs krb5kdc[14064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.130@KRBTEST.COM
I20260924 00:56:35.586566 14585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:35.571763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37114 (local address 127.8.64.130:33165)
0924 00:56:35.572348 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:35.572356 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:35.572382 (+ 26us) server_negotiation.cc:407] Connection header received
0924 00:56:35.572451 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:35.572457 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:35.572660 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:35.572797 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:35.573936 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:35.575045 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:35.576309 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:35.576844 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:35.576939 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:35.581190 (+ 4251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:35.581200 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:35.581215 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:35.581248 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:35.584116 (+ 2868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:35.584687 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:35.584693 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:35.584695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:35.584826 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:35.585133 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:35.585136 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:35.585138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:35.585448 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:35.585658 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:35.585943 (+ 285us) server_negotiation.cc:299] Negotiation successful
0924 00:56:35.586123 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":321,"threads_started":1}
I20260924 00:56:35.587852 14427 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:56:35.669090 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 14304
I20260924 00:56:35.685154 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 14437
Sep 24 00:56:35 dist-test-slave-mmhs krb5kdc[14064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.131@KRBTEST.COM
I20260924 00:56:35.700220 14587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:35.686159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51484 (local address 127.8.64.131:37059)
0924 00:56:35.686826 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:35.686835 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:35.686873 (+ 38us) server_negotiation.cc:407] Connection header received
0924 00:56:35.686933 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:35.686939 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:35.687148 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:35.687330 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:35.688375 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:35.689384 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:35.690711 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:35.691257 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:35.691391 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:35.695222 (+ 3831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:35.695232 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:35.695253 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:35.695286 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:35.697806 (+ 2520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:35.698474 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:35.698480 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:35.698483 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:35.698596 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:35.698919 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:35.698921 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:35.698923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:35.699226 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:35.699452 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:35.699674 (+ 222us) server_negotiation.cc:299] Negotiation successful
0924 00:56:35.699931 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":376,"threads_started":1}
I20260924 00:56:35.701509 14560 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:56:35.783809 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 14437
I20260924 00:56:35.799525 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 14077
I20260924 00:56:35.800714 14139 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:56:35.901196 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 14077
2026-09-24T00:56:35Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3928 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:56:36 dist-test-slave-mmhs krb5kdc[14592](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:56:36 dist-test-slave-mmhs krb5kdc[14592](info): set up 2 sockets
Sep 24 00:56:36 dist-test-slave-mmhs krb5kdc[14592](info): commencing operation
krb5kdc: starting...
W20260924 00:56:37.000789 8450 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.008s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Sep 24 00:56:37 dist-test-slave-mmhs krb5kdc[14592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:56:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:56:37Z Disabled control of system clock
WARNING: no policy specified for oryx/127.8.64.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.64.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.64.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.190.keytab.
Entry for principal oryx/127.8.64.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.190.keytab.
I20260924 00:56:37.213884 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:44743
--webserver_interface=127.8.64.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.190.keytab
--principal=oryx/127.8.64.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:41079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:44743
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:56:37.526827 14605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:37.527107 14605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:37.527285 14605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:37.534332 14605 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:56:37.534482 14605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:37.534583 14605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:37.534690 14605 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:56:37.534780 14605 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:56:37.544555 14605 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:41079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:44743
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.190.keytab
--principal=oryx/127.8.64.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:44743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:37.545873 14605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:37.547174 14605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:37.558910 14610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:37.558848 14613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:37.558950 14611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:37.559731 14605 server_base.cc:1034] running on GCE node
I20260924 00:56:37.560452 14605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:37.561924 14605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:37.563153 14605 hybrid_clock.cc:648] HybridClock initialized: now 1790211397563115 us; error 53 us; skew 500 ppm
I20260924 00:56:37.563513 14605 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:37 dist-test-slave-mmhs krb5kdc[14592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211397, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.64.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:37.566686 14605 init.cc:377] Logged in from keytab as oryx/127.8.64.190@KRBTEST.COM (short username oryx)
I20260924 00:56:37.568104 14605 webserver.cc:460] Webserver started at http://127.8.64.190:46817/ using document root <none> and password file <none>
I20260924 00:56:37.568799 14605 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:37.568953 14605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:37.569253 14605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:56:37.572315 14605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "db882bdd18eb4243928039daa8c49303"
format_stamp: "Formatted at 2026-09-24 00:56:37 on dist-test-slave-mmhs"
server_key: "6c2b56e93071bcdc11cd30360b9eb934"
server_key_iv: "f9cd4e3ae84faf10f09f2ae0a24caffd"
server_key_version: "encryptionkey@0"
I20260924 00:56:37.573076 14605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "db882bdd18eb4243928039daa8c49303"
format_stamp: "Formatted at 2026-09-24 00:56:37 on dist-test-slave-mmhs"
server_key: "6c2b56e93071bcdc11cd30360b9eb934"
server_key_iv: "f9cd4e3ae84faf10f09f2ae0a24caffd"
server_key_version: "encryptionkey@0"
I20260924 00:56:37.578176 14605 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260924 00:56:37.582084 14620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:37.583546 14605 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260924 00:56:37.583742 14605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "db882bdd18eb4243928039daa8c49303"
format_stamp: "Formatted at 2026-09-24 00:56:37 on dist-test-slave-mmhs"
server_key: "6c2b56e93071bcdc11cd30360b9eb934"
server_key_iv: "f9cd4e3ae84faf10f09f2ae0a24caffd"
server_key_version: "encryptionkey@0"
I20260924 00:56:37.583946 14605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:37.605119 14605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:37.609103 14605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:37.609401 14605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:37.627610 14605 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.190:44743
I20260924 00:56:37.627630 14671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.190:44743 every 8 connection(s)
I20260924 00:56:37.628931 14605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 00:56:37.632550 14672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:37.632571 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 14605
I20260924 00:56:37.632779 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 00:56:37.633419 8450 external_mini_cluster.cc:1428] Setting key 46017cc31a5b96f63be71a1c21b4931e
I20260924 00:56:37.646831 14672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303: Bootstrap starting.
Sep 24 00:56:37 dist-test-slave-mmhs krb5kdc[14592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM
I20260924 00:56:37.650158 14672 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:37.651506 14672 log.cc:824] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303: Log is configured to *not* fsync() on all Append() calls
I20260924 00:56:37.654219 14672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303: No bootstrap required, opened a new log
I20260924 00:56:37.657004 14675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:37.635430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39598 (local address 127.8.64.190:44743)
0924 00:56:37.636249 (+ 819us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:37.636273 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:37.636328 (+ 55us) server_negotiation.cc:407] Connection header received
0924 00:56:37.639105 (+ 2777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:37.639134 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:37.639665 (+ 531us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:37.640108 (+ 443us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:37.641355 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:37.643394 (+ 2039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:37.644627 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:37.645637 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:37.645806 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:37.649747 (+ 3941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:37.649757 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:37.649776 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:37.649814 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:37.653087 (+ 3273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:37.653767 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:37.653784 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:37.653789 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:37.653959 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:37.654360 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:37.654367 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:37.654371 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:37.654862 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:37.655176 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:37.655603 (+ 427us) server_negotiation.cc:299] Negotiation successful
0924 00:56:37.656056 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":336,"threads_started":1}
I20260924 00:56:37.661926 14672 raft_consensus.cc:348] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } }
I20260924 00:56:37.662287 14672 raft_consensus.cc:374] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:37.662437 14672 raft_consensus.cc:729] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db882bdd18eb4243928039daa8c49303, State: Initialized, Role: FOLLOWER
I20260924 00:56:37.663062 14672 consensus_queue.cc:260] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } }
I20260924 00:56:37.663290 14672 raft_consensus.cc:388] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:56:37.663441 14672 raft_consensus.cc:482] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:56:37.663625 14672 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:37.666078 14672 raft_consensus.cc:504] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } }
I20260924 00:56:37.666649 14672 leader_election.cc:302] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db882bdd18eb4243928039daa8c49303; no voters:
I20260924 00:56:37.667225 14672 leader_election.cc:288] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:56:37.667343 14677 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:56:37.668058 14677 raft_consensus.cc:686] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [term 1 LEADER]: Becoming Leader. State: Replica: db882bdd18eb4243928039daa8c49303, State: Running, Role: LEADER
I20260924 00:56:37.668596 14677 consensus_queue.cc:237] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } }
I20260924 00:56:37.669207 14672 sys_catalog.cc:564] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:56:37.671360 14678 sys_catalog.cc:455] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db882bdd18eb4243928039daa8c49303" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } }
I20260924 00:56:37.671715 14678 sys_catalog.cc:458] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [sys.catalog]: This master's current role is: LEADER
I20260924 00:56:37.671835 14679 sys_catalog.cc:455] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db882bdd18eb4243928039daa8c49303. Latest consensus state: current_term: 1 leader_uuid: "db882bdd18eb4243928039daa8c49303" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } }
I20260924 00:56:37.672250 14679 sys_catalog.cc:458] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [sys.catalog]: This master's current role is: LEADER
I20260924 00:56:37.675103 14682 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:56:37.681450 14682 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:56:37.690021 14682 catalog_manager.cc:1347] Generated new cluster ID: d3b5b249561e46889e63beb4a75853a8
I20260924 00:56:37.690132 14682 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:56:37.700734 14682 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:56:37.702592 14682 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:56:37.710960 14682 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303: Generated new TSK 0
I20260924 00:56:37.711795 14682 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.8.64.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.64.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.64.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.129.keytab.
Entry for principal oryx/127.8.64.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.129.keytab.
I20260924 00:56:37.769768 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.129:0
--local_ip_for_outbound_sockets=127.8.64.129
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:44743
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.129.keytab
--principal=oryx/127.8.64.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:41079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:56:38.098246 14698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:38.098630 14698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:38.098732 14698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:38.105878 14698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:38.105988 14698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:38.106101 14698 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.129
I20260924 00:56:38.115797 14698 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:41079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.129.keytab
--principal=oryx/127.8.64.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:44743
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.129
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:38.117138 14698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:38.119119 14698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:38.133515 14703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:38.133484 14706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:38.133718 14704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:38.134548 14698 server_base.cc:1034] running on GCE node
I20260924 00:56:38.135128 14698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:38.136075 14698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:38.137312 14698 hybrid_clock.cc:648] HybridClock initialized: now 1790211398137226 us; error 73 us; skew 500 ppm
I20260924 00:56:38.137606 14698 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:38 dist-test-slave-mmhs krb5kdc[14592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211398, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.64.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:38.140818 14698 init.cc:377] Logged in from keytab as oryx/127.8.64.129@KRBTEST.COM (short username oryx)
I20260924 00:56:38.142489 14698 webserver.cc:460] Webserver started at http://127.8.64.129:37795/ using document root <none> and password file <none>
I20260924 00:56:38.143195 14698 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:38.143349 14698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:38.143663 14698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:56:38.146901 14698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-0/data/instance:
uuid: "22c762dcd2e94b6f91721a9571613065"
format_stamp: "Formatted at 2026-09-24 00:56:38 on dist-test-slave-mmhs"
server_key: "dcda79cd347d800d081d192118aac951"
server_key_iv: "bf54d53d6921c835aa6f5e295b122cfd"
server_key_version: "encryptionkey@0"
I20260924 00:56:38.147689 14698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance:
uuid: "22c762dcd2e94b6f91721a9571613065"
format_stamp: "Formatted at 2026-09-24 00:56:38 on dist-test-slave-mmhs"
server_key: "dcda79cd347d800d081d192118aac951"
server_key_iv: "bf54d53d6921c835aa6f5e295b122cfd"
server_key_version: "encryptionkey@0"
I20260924 00:56:38.152966 14698 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260924 00:56:38.156582 14713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:38.157972 14698 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260924 00:56:38.158149 14698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-0/wal
uuid: "22c762dcd2e94b6f91721a9571613065"
format_stamp: "Formatted at 2026-09-24 00:56:38 on dist-test-slave-mmhs"
server_key: "dcda79cd347d800d081d192118aac951"
server_key_iv: "bf54d53d6921c835aa6f5e295b122cfd"
server_key_version: "encryptionkey@0"
I20260924 00:56:38.158442 14698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:38.178362 14698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:38.182662 14698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:38.182951 14698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:38.183923 14698 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:56:38.185274 14698 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:56:38.185379 14698 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:38.185523 14698 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:56:38.185595 14698 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:38.218808 14698 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.129:44385
I20260924 00:56:38.218894 14825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.129:44385 every 8 connection(s)
I20260924 00:56:38.220115 14698 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
I20260924 00:56:38.220917 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 14698
I20260924 00:56:38.221138 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance
I20260924 00:56:38.221854 8450 external_mini_cluster.cc:1428] Setting key f6f053e71e57aa272237330b3280e37b
Sep 24 00:56:38 dist-test-slave-mmhs krb5kdc[14592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211398, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.64.129@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM
I20260924 00:56:38.241129 14828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:38.222952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:60705 (local address 127.8.64.190:44743)
0924 00:56:38.223966 (+ 1014us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:38.223974 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:38.224531 (+ 557us) server_negotiation.cc:407] Connection header received
0924 00:56:38.226353 (+ 1822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:38.226361 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:38.226454 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:38.226571 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:38.228056 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:38.229207 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:38.230866 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:38.231291 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:38.231430 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:38.235583 (+ 4153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:38.235588 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:38.235591 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:38.235622 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:38.238140 (+ 2518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:38.238882 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:38.238889 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:38.238894 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:38.239012 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:38.239432 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:38.239437 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:38.239440 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:38.239697 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:38.239849 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:38.240541 (+ 692us) server_negotiation.cc:299] Negotiation successful
0924 00:56:38.240770 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":905,"thread_start_us":821,"threads_started":1}
I20260924 00:56:38.242168 14829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:38.223455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44743 (local address 127.8.64.129:60705)
0924 00:56:38.224357 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:38.224401 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:38.226061 (+ 1660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:38.226790 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:38.226798 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:38.227303 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:38.227874 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:38.227891 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:38.229370 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:38.229374 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:38.230715 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:38.230726 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:38.231522 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:38.231528 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:38.231866 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:38.232579 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:38.232617 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:38.235361 (+ 2744us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:38.238312 (+ 2951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:38.238322 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:38.238341 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:38.238764 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:38.239119 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:38.239125 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:38.239129 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:38.239316 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:38.239822 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:38.239830 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:38.240163 (+ 333us) client_negotiation.cc:769] Sending connection context
0924 00:56:38.240440 (+ 277us) client_negotiation.cc:240] Negotiation successful
0924 00:56:38.240957 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":407,"threads_started":1}
WARNING: no policy specified for oryx/127.8.64.130@KRBTEST.COM; defaulting to no policy
I20260924 00:56:38.243796 14826 heartbeater.cc:347] Connected to a master server at 127.8.64.190:44743
I20260924 00:56:38.244071 14826 heartbeater.cc:464] Registering TS with master...
I20260924 00:56:38.244864 14826 heartbeater.cc:511] Master 127.8.64.190:44743 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.64.130@KRBTEST.COM" created.
I20260924 00:56:38.247293 14637 ts_manager.cc:194] Registered new tserver with Master: 22c762dcd2e94b6f91721a9571613065 (127.8.64.129:44385)
I20260924 00:56:38.249504 14637 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.64.129@KRBTEST.COM'} at 127.8.64.129:60705
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.64.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.130.keytab.
Entry for principal oryx/127.8.64.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.130.keytab.
I20260924 00:56:38.274734 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.130:0
--local_ip_for_outbound_sockets=127.8.64.130
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:44743
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.130.keytab
--principal=oryx/127.8.64.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:41079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:56:38.600073 14832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:38.600405 14832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:38.600581 14832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:38.607972 14832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:38.608155 14832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:38.608343 14832 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.130
I20260924 00:56:38.618324 14832 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:41079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.130.keytab
--principal=oryx/127.8.64.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:44743
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.130
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:38.619685 14832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:38.621006 14832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:38.634168 14837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:38.634169 14838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:38.634729 14840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:38.634975 14832 server_base.cc:1034] running on GCE node
I20260924 00:56:38.635646 14832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:38.636644 14832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:38.637892 14832 hybrid_clock.cc:648] HybridClock initialized: now 1790211398637848 us; error 34 us; skew 500 ppm
I20260924 00:56:38.638268 14832 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:38 dist-test-slave-mmhs krb5kdc[14592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211398, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.64.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:38.641656 14832 init.cc:377] Logged in from keytab as oryx/127.8.64.130@KRBTEST.COM (short username oryx)
I20260924 00:56:38.643225 14832 webserver.cc:460] Webserver started at http://127.8.64.130:33341/ using document root <none> and password file <none>
I20260924 00:56:38.643980 14832 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:38.644120 14832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:38.644424 14832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:56:38.647338 14832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-1/data/instance:
uuid: "72f23ce899264c60b7b9818e9cc2797c"
format_stamp: "Formatted at 2026-09-24 00:56:38 on dist-test-slave-mmhs"
server_key: "f2bc4483bcddb6bb9f658e8ffc69dd48"
server_key_iv: "20e951698e883a24b9a2bd3045048b04"
server_key_version: "encryptionkey@0"
I20260924 00:56:38.648082 14832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance:
uuid: "72f23ce899264c60b7b9818e9cc2797c"
format_stamp: "Formatted at 2026-09-24 00:56:38 on dist-test-slave-mmhs"
server_key: "f2bc4483bcddb6bb9f658e8ffc69dd48"
server_key_iv: "20e951698e883a24b9a2bd3045048b04"
server_key_version: "encryptionkey@0"
I20260924 00:56:38.653064 14832 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260924 00:56:38.656664 14847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:38.657940 14832 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260924 00:56:38.658109 14832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-1/wal
uuid: "72f23ce899264c60b7b9818e9cc2797c"
format_stamp: "Formatted at 2026-09-24 00:56:38 on dist-test-slave-mmhs"
server_key: "f2bc4483bcddb6bb9f658e8ffc69dd48"
server_key_iv: "20e951698e883a24b9a2bd3045048b04"
server_key_version: "encryptionkey@0"
I20260924 00:56:38.658366 14832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:38.677322 14832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:38.681461 14832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:38.681777 14832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:38.682778 14832 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:56:38.684099 14832 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:56:38.684203 14832 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:38.684355 14832 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:56:38.684429 14832 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:38.717463 14832 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.130:44619
I20260924 00:56:38.717509 14960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.130:44619 every 8 connection(s)
I20260924 00:56:38.718760 14832 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
I20260924 00:56:38.725544 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 14832
I20260924 00:56:38.725682 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance
I20260924 00:56:38.726218 8450 external_mini_cluster.cc:1428] Setting key d8966ea996f79c91b54fa4a5d643f762
Sep 24 00:56:38 dist-test-slave-mmhs krb5kdc[14592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211398, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.64.130@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM
I20260924 00:56:38.740039 14828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:38.721154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:46679 (local address 127.8.64.190:44743)
0924 00:56:38.721316 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:38.721320 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:38.722592 (+ 1272us) server_negotiation.cc:407] Connection header received
0924 00:56:38.724340 (+ 1748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:38.724344 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:38.724409 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:38.724502 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:38.726021 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:38.727025 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:38.728711 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:38.729464 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:38.729606 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:38.733171 (+ 3565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:38.733180 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:38.733188 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:38.733235 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:38.736695 (+ 3460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:38.737612 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:38.737615 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:38.737618 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:38.737736 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:38.738221 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:38.738228 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:38.738233 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:38.738631 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:38.738756 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:38.739615 (+ 859us) server_negotiation.cc:299] Negotiation successful
0924 00:56:38.739784 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260924 00:56:38.741660 14963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:38.721582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44743 (local address 127.8.64.130:46679)
0924 00:56:38.722435 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:38.722480 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:38.724101 (+ 1621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:38.724699 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:38.724706 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:38.725182 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:38.725851 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:38.725871 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:38.727186 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:38.727190 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:38.728570 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:38.728582 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:38.729670 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:38.729674 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:38.729871 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:38.730577 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:38.730602 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:38.732935 (+ 2333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:38.736836 (+ 3901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:38.736848 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:38.736868 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:38.737498 (+ 630us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:38.737842 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:38.737849 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:38.737855 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:38.738091 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:38.738781 (+ 690us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:38.738792 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:38.739248 (+ 456us) client_negotiation.cc:769] Sending connection context
0924 00:56:38.739562 (+ 314us) client_negotiation.cc:240] Negotiation successful
0924 00:56:38.740142 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":354,"threads_started":1}
I20260924 00:56:38.743263 14961 heartbeater.cc:347] Connected to a master server at 127.8.64.190:44743
I20260924 00:56:38.743594 14961 heartbeater.cc:464] Registering TS with master...
I20260924 00:56:38.744475 14961 heartbeater.cc:511] Master 127.8.64.190:44743 requested a full tablet report, sending...
I20260924 00:56:38.746200 14637 ts_manager.cc:194] Registered new tserver with Master: 72f23ce899264c60b7b9818e9cc2797c (127.8.64.130:44619)
I20260924 00:56:38.747540 14637 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.64.130@KRBTEST.COM'} at 127.8.64.130:46679
WARNING: no policy specified for oryx/127.8.64.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.64.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.64.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.131.keytab.
Entry for principal oryx/127.8.64.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.131.keytab.
I20260924 00:56:38.782835 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.131:0
--local_ip_for_outbound_sockets=127.8.64.131
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:44743
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.131.keytab
--principal=oryx/127.8.64.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:41079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:56:39.104740 14966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:39.105068 14966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:39.105247 14966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:39.112416 14966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:39.112582 14966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:39.112777 14966 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.131
I20260924 00:56:39.122272 14966 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:41079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.131.keytab
--principal=oryx/127.8.64.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:44743
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.131
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:39.123574 14966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:39.124859 14966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:39.137969 14971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:39.138051 14972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:39.138082 14974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:39.139248 14966 server_base.cc:1034] running on GCE node
I20260924 00:56:39.139905 14966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:39.140813 14966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:39.142064 14966 hybrid_clock.cc:648] HybridClock initialized: now 1790211399142015 us; error 60 us; skew 500 ppm
I20260924 00:56:39.142494 14966 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:39 dist-test-slave-mmhs krb5kdc[14592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211399, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.64.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:39.145793 14966 init.cc:377] Logged in from keytab as oryx/127.8.64.131@KRBTEST.COM (short username oryx)
I20260924 00:56:39.147356 14966 webserver.cc:460] Webserver started at http://127.8.64.131:37655/ using document root <none> and password file <none>
I20260924 00:56:39.148053 14966 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:39.148198 14966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:39.148491 14966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:56:39.151403 14966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-2/data/instance:
uuid: "99ac8d2d4cd2408bab694ed9594ef526"
format_stamp: "Formatted at 2026-09-24 00:56:39 on dist-test-slave-mmhs"
server_key: "637afeb5b51098d6c34178ac453d100e"
server_key_iv: "9b429a3a3ce897ac4380142a7510c764"
server_key_version: "encryptionkey@0"
I20260924 00:56:39.152153 14966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance:
uuid: "99ac8d2d4cd2408bab694ed9594ef526"
format_stamp: "Formatted at 2026-09-24 00:56:39 on dist-test-slave-mmhs"
server_key: "637afeb5b51098d6c34178ac453d100e"
server_key_iv: "9b429a3a3ce897ac4380142a7510c764"
server_key_version: "encryptionkey@0"
I20260924 00:56:39.157172 14966 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260924 00:56:39.160508 14981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:39.161897 14966 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260924 00:56:39.162065 14966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-2/wal
uuid: "99ac8d2d4cd2408bab694ed9594ef526"
format_stamp: "Formatted at 2026-09-24 00:56:39 on dist-test-slave-mmhs"
server_key: "637afeb5b51098d6c34178ac453d100e"
server_key_iv: "9b429a3a3ce897ac4380142a7510c764"
server_key_version: "encryptionkey@0"
I20260924 00:56:39.162335 14966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:39.184484 14966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:39.189621 14966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:39.190008 14966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:39.191879 14966 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:56:39.193734 14966 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:56:39.193843 14966 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:39.193961 14966 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:56:39.194025 14966 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:39.227543 14966 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.131:33651
I20260924 00:56:39.227588 15093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.131:33651 every 8 connection(s)
I20260924 00:56:39.228703 14966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
I20260924 00:56:39.233291 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 14966
I20260924 00:56:39.233474 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance
I20260924 00:56:39.234019 8450 external_mini_cluster.cc:1428] Setting key 4950d49f9f3ab2fce96b52866f173a24
Sep 24 00:56:39 dist-test-slave-mmhs krb5kdc[14592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211399, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.64.131@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM
I20260924 00:56:39.248785 14828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:39.231280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:42003 (local address 127.8.64.190:44743)
0924 00:56:39.231451 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:39.231455 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:39.232469 (+ 1014us) server_negotiation.cc:407] Connection header received
0924 00:56:39.234968 (+ 2499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:39.234974 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:39.235060 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:39.235177 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:39.236776 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:39.237589 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:39.239146 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:39.239525 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:39.239652 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:39.243012 (+ 3360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:39.243018 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:39.243021 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:39.243057 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:39.245641 (+ 2584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:39.246422 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:39.246427 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:39.246429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:39.246558 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:39.246934 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:39.246937 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:39.246939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:39.247270 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:39.247453 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:39.248204 (+ 751us) server_negotiation.cc:299] Negotiation successful
0924 00:56:39.248420 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260924 00:56:39.249698 15096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:39.231615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44743 (local address 127.8.64.131:42003)
0924 00:56:39.232305 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:39.232347 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:39.234673 (+ 2326us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:39.235457 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:39.235468 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:39.236070 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:39.236618 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:39.236635 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:39.237750 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:39.237754 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:39.239007 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:39.239018 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:39.239744 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:39.239752 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:39.239973 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:39.240569 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:39.240594 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:39.242794 (+ 2200us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:39.245787 (+ 2993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:39.245794 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:39.245807 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:39.246210 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:39.246656 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:39.246659 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:39.246661 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:39.246834 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:39.247447 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:39.247454 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:39.247826 (+ 372us) client_negotiation.cc:769] Sending connection context
0924 00:56:39.248084 (+ 258us) client_negotiation.cc:240] Negotiation successful
0924 00:56:39.248535 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":272,"threads_started":1}
I20260924 00:56:39.251024 15094 heartbeater.cc:347] Connected to a master server at 127.8.64.190:44743
I20260924 00:56:39.251323 15094 heartbeater.cc:464] Registering TS with master...
I20260924 00:56:39.252008 15094 heartbeater.cc:511] Master 127.8.64.190:44743 requested a full tablet report, sending...
I20260924 00:56:39.252558 14826 heartbeater.cc:503] Master 127.8.64.190:44743 was elected leader, sending a full tablet report...
I20260924 00:56:39.253502 14637 ts_manager.cc:194] Registered new tserver with Master: 99ac8d2d4cd2408bab694ed9594ef526 (127.8.64.131:33651)
I20260924 00:56:39.254457 14637 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.64.131@KRBTEST.COM'} at 127.8.64.131:42003
I20260924 00:56:39.261741 8450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.8.64.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.64.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.64.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.189.keytab.
Entry for principal oryx/127.8.64.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.189.keytab.
I20260924 00:56:39.309414 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.189:39347
--webserver_interface=127.8.64.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.189.keytab
--principal=oryx/127.8.64.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:41079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:44743,127.8.64.189:39347
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:56:39.637029 15099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:39.637300 15099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:39.637387 15099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:39.644528 15099 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:56:39.644634 15099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:39.644681 15099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:39.644737 15099 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:56:39.644779 15099 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:56:39.654616 15099 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:41079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:44743,127.8.64.189:39347
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.189.keytab
--principal=oryx/127.8.64.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.189:39347
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.64.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
W20260924 00:56:39.654946 15099 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.8.64.190:44743,127.8.64.189:39347'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260924 00:56:39.655946 15099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:39.657174 15099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:39.670001 15104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:39.670056 15105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:39.670092 15107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:39.671263 15099 server_base.cc:1034] running on GCE node
I20260924 00:56:39.672086 15099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:39.673482 15099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:39.674695 15099 hybrid_clock.cc:648] HybridClock initialized: now 1790211399674625 us; error 42 us; skew 500 ppm
I20260924 00:56:39.675041 15099 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:39 dist-test-slave-mmhs krb5kdc[14592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211399, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.64.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:39.683153 15099 init.cc:377] Logged in from keytab as oryx/127.8.64.189@KRBTEST.COM (short username oryx)
I20260924 00:56:39.684661 15099 webserver.cc:460] Webserver started at http://127.8.64.189:33837/ using document root <none> and password file <none>
I20260924 00:56:39.685367 15099 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:39.685457 15099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:39.685720 15099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:56:39.688885 15099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-1/data/instance:
uuid: "1d362bb4f4dd41adaee4592166347b34"
format_stamp: "Formatted at 2026-09-24 00:56:39 on dist-test-slave-mmhs"
server_key: "66eddb1f71e03689da3792060cdd6697"
server_key_iv: "1b93944cfc81500e431865d72f272cfd"
server_key_version: "encryptionkey@0"
I20260924 00:56:39.689685 15099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-1/wal/instance:
uuid: "1d362bb4f4dd41adaee4592166347b34"
format_stamp: "Formatted at 2026-09-24 00:56:39 on dist-test-slave-mmhs"
server_key: "66eddb1f71e03689da3792060cdd6697"
server_key_iv: "1b93944cfc81500e431865d72f272cfd"
server_key_version: "encryptionkey@0"
I20260924 00:56:39.694828 15099 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260924 00:56:39.698215 15114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:39.699553 15099 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260924 00:56:39.699738 15099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-1/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-1/wal
uuid: "1d362bb4f4dd41adaee4592166347b34"
format_stamp: "Formatted at 2026-09-24 00:56:39 on dist-test-slave-mmhs"
server_key: "66eddb1f71e03689da3792060cdd6697"
server_key_iv: "1b93944cfc81500e431865d72f272cfd"
server_key_version: "encryptionkey@0"
I20260924 00:56:39.699977 15099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:39.726104 15099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:39.730206 15099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:39.730502 15099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:39.748955 15099 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.189:39347
I20260924 00:56:39.748962 15165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.189:39347 every 8 connection(s)
I20260924 00:56:39.750597 15099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-1/data/info.pb
I20260924 00:56:39.750978 14961 heartbeater.cc:503] Master 127.8.64.190:44743 was elected leader, sending a full tablet report...
I20260924 00:56:39.754665 15166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:39.760828 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 15099
I20260924 00:56:39.769675 15166 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 00:56:39 dist-test-slave-mmhs krb5kdc[14592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.64.189@KRBTEST.COM
I20260924 00:56:39.795671 15169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:39.763787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37346 (local address 127.8.64.189:39347)
0924 00:56:39.768931 (+ 5144us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:39.768943 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:39.768992 (+ 49us) server_negotiation.cc:407] Connection header received
0924 00:56:39.771110 (+ 2118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:39.771133 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:39.771654 (+ 521us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:39.772112 (+ 458us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:39.773573 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:39.775365 (+ 1792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:39.777969 (+ 2604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:39.779378 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:39.780476 (+ 1098us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:39.786639 (+ 6163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:39.786655 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:39.786684 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:39.786745 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:39.791124 (+ 4379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:39.792007 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:39.792018 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:39.792031 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:39.792261 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:39.792731 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:39.792738 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:39.792744 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:39.793361 (+ 617us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:39.793690 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:39.794167 (+ 477us) server_negotiation.cc:299] Negotiation successful
0924 00:56:39.794654 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4878,"thread_start_us":4549,"threads_started":1}
Sep 24 00:56:39 dist-test-slave-mmhs krb5kdc[14592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211399, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.64.189@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM
I20260924 00:56:39.806339 15171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:39.774941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44743 (local address 127.0.0.1:39606)
0924 00:56:39.788790 (+ 13849us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:39.788847 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:39.789033 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:39.789446 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:39.789456 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:39.790690 (+ 1234us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:39.791207 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:39.791225 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:39.792524 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:39.792531 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:39.794736 (+ 2205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:39.794755 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:39.796127 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:39.796133 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:39.796356 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:39.797078 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:39.797106 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:39.799456 (+ 2350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:39.803666 (+ 4210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:39.803675 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:39.803680 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:39.804137 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:39.804597 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:39.804601 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:39.804603 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:39.804732 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:39.805488 (+ 756us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:39.805493 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:39.805627 (+ 134us) client_negotiation.cc:769] Sending connection context
0924 00:56:39.805770 (+ 143us) client_negotiation.cc:240] Negotiation successful
0924 00:56:39.806005 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13612,"thread_start_us":13178,"threads_started":1}
I20260924 00:56:39.806701 15170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:39.774890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39606 (local address 127.8.64.190:44743)
0924 00:56:39.788379 (+ 13489us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:39.788385 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:39.789148 (+ 763us) server_negotiation.cc:407] Connection header received
0924 00:56:39.789202 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:39.789206 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:39.789270 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:39.789359 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:39.791411 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:39.792391 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:39.794939 (+ 2548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:39.795924 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:39.796067 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:39.799681 (+ 3614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:39.799690 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:39.799698 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:39.799747 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:39.803540 (+ 3793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:39.804309 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:39.804316 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:39.804322 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:39.804483 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:39.804846 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:39.804854 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:39.804859 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:39.805342 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:39.805516 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:39.805935 (+ 419us) server_negotiation.cc:299] Negotiation successful
0924 00:56:39.806200 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13353,"thread_start_us":12929,"threads_started":1}
I20260924 00:56:39.808336 15166 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.64.189" port: 39347 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 00:56:39 dist-test-slave-mmhs krb5kdc[14592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211399, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.64.189@KRBTEST.COM for oryx/127.8.64.189@KRBTEST.COM
I20260924 00:56:39.823056 15171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:39.809025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:39347 (local address 127.0.0.1:37354)
0924 00:56:39.809206 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:39.809224 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:39.809317 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:39.809768 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:39.809771 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:39.810090 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:39.810216 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:39.810224 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:39.811853 (+ 1629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:39.811857 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:39.812708 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:39.812718 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:39.813831 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:39.813840 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:39.814005 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:39.814611 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:39.814627 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:39.816572 (+ 1945us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:39.820787 (+ 4215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:39.820790 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:39.820793 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:39.821170 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:39.821558 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:39.821561 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:39.821564 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:39.821682 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:39.822376 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:39.822380 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:39.822506 (+ 126us) client_negotiation.cc:769] Sending connection context
0924 00:56:39.822612 (+ 106us) client_negotiation.cc:240] Negotiation successful
0924 00:56:39.822804 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":1}
I20260924 00:56:39.823493 15169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:39.809269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37354 (local address 127.8.64.189:39347)
0924 00:56:39.809454 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:39.809462 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:39.809482 (+ 20us) server_negotiation.cc:407] Connection header received
0924 00:56:39.809543 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:39.809550 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:39.809631 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:39.809734 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:39.810362 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:39.811698 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:39.812848 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:39.813677 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:39.813832 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:39.816744 (+ 2912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:39.816752 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:39.816759 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:39.816801 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:39.820639 (+ 3838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:39.821289 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:39.821296 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:39.821301 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:39.821447 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:39.821778 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:39.821786 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:39.821791 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:39.822216 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:39.822407 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:39.822749 (+ 342us) server_negotiation.cc:299] Negotiation successful
0924 00:56:39.823015 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":54}
I20260924 00:56:39.828245 15166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34: Bootstrap starting.
I20260924 00:56:39.831406 15166 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:39.832448 15166 log.cc:824] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34: Log is configured to *not* fsync() on all Append() calls
I20260924 00:56:39.835126 15166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34: No bootstrap required, opened a new log
I20260924 00:56:39.842419 15166 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } peers { permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 39347 } }
I20260924 00:56:39.842739 15166 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:39.842837 15166 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d362bb4f4dd41adaee4592166347b34, State: Initialized, Role: FOLLOWER
I20260924 00:56:39.843464 15166 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } peers { permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 39347 } }
I20260924 00:56:39.844404 15173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } peers { permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 39347 } } }
I20260924 00:56:39.844815 15173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:56:39.845472 15166 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:56:39.856372 15099 master_runner.cc:428] Detected that this master 1d362bb4f4dd41adaee4592166347b34 is joining an existing cluster
I20260924 00:56:39.856482 15099 master_runner.cc:432] Initiating AddMaster RPC to add 127.8.64.189:39347
W20260924 00:56:39.861092 15184 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 00:56:39.861312 15184 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 00:56:39.883292 15170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:39.864858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39618 (local address 127.8.64.190:44743)
0924 00:56:39.865069 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:39.865074 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:39.866378 (+ 1304us) server_negotiation.cc:407] Connection header received
0924 00:56:39.866669 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:39.866678 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:39.866808 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:39.866896 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:39.868764 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:39.870228 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:39.873032 (+ 2804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:39.873869 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:39.874025 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:39.876807 (+ 2782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:39.876813 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:39.876816 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:39.876849 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:39.880322 (+ 3473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:39.880946 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:39.880953 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:39.880957 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:39.881121 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:39.881601 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:39.881607 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:39.881611 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:39.881960 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:39.882123 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:39.882692 (+ 569us) server_negotiation.cc:299] Negotiation successful
0924 00:56:39.882942 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260924 00:56:39.883726 15190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:39.864310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44743 (local address 127.0.0.1:39618)
0924 00:56:39.866261 (+ 1951us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:39.866280 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:39.866486 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:39.867164 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:39.867172 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:39.868073 (+ 901us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:39.868553 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:39.868570 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:39.870488 (+ 1918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:39.870496 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:39.872789 (+ 2293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:39.872811 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:39.874346 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:39.874359 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:39.874615 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:39.875392 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:39.875424 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:39.876591 (+ 1167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:39.880460 (+ 3869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:39.880467 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:39.880471 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:39.880816 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:39.881236 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:39.881241 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:39.881245 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:39.881469 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:39.882082 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:39.882087 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:39.882333 (+ 246us) client_negotiation.cc:769] Sending connection context
0924 00:56:39.882585 (+ 252us) client_negotiation.cc:240] Negotiation successful
0924 00:56:39.883203 (+ 618us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1801,"thread_start_us":1165,"threads_started":1}
I20260924 00:56:39.897472 15191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:39.866244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:39347 (local address 127.0.0.1:37364)
0924 00:56:39.873390 (+ 7146us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:39.873409 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:39.873714 (+ 305us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:39.874133 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:39.874138 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:39.874794 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:39.875191 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:39.875204 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:39.876716 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:39.876720 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:39.878252 (+ 1532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:39.878266 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:39.879133 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:39.879141 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:39.879326 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:39.879914 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:39.879938 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:39.880980 (+ 1042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:39.884875 (+ 3895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:39.884880 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:39.884884 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:39.885430 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:39.894803 (+ 9373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:39.894815 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:39.894822 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:39.895039 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:39.896000 (+ 961us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:39.896008 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:39.896220 (+ 212us) client_negotiation.cc:769] Sending connection context
0924 00:56:39.896389 (+ 169us) client_negotiation.cc:240] Negotiation successful
0924 00:56:39.896779 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6988,"thread_start_us":7152,"threads_started":1}
I20260924 00:56:39.898900 15169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:39.865645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37364 (local address 127.8.64.189:39347)
0924 00:56:39.865853 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:39.865859 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:39.873474 (+ 7615us) server_negotiation.cc:407] Connection header received
0924 00:56:39.873832 (+ 358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:39.873838 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:39.873901 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:39.873958 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:39.875356 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:39.876541 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:39.878459 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:39.878935 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:39.879030 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:39.881171 (+ 2141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:39.881180 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:39.881187 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:39.881224 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:39.884736 (+ 3512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:39.885550 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:39.885554 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:39.885557 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:39.894579 (+ 9022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:39.895249 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:39.895257 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:39.895262 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:39.895729 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:39.895938 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:39.898021 (+ 2083us) server_negotiation.cc:299] Negotiation successful
0924 00:56:39.898387 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Sep 24 00:56:39 dist-test-slave-mmhs krb5kdc[14592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211397, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.64.190@KRBTEST.COM for oryx/127.8.64.189@KRBTEST.COM
I20260924 00:56:39.923902 15169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:39.900318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37378 (local address 127.8.64.189:39347)
0924 00:56:39.900840 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:39.901300 (+ 460us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:39.902025 (+ 725us) server_negotiation.cc:407] Connection header received
0924 00:56:39.902476 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:39.902485 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:39.902590 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:39.902671 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:39.904983 (+ 2312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:39.906323 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:39.908709 (+ 2386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:39.909613 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:39.909733 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:39.914840 (+ 5107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:39.914849 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:39.914857 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:39.914922 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:39.919538 (+ 4616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:39.920639 (+ 1101us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:39.920648 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:39.920654 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:39.920815 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:39.921414 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:39.921422 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:39.921428 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:39.921879 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:39.922026 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:39.922941 (+ 915us) server_negotiation.cc:299] Negotiation successful
0924 00:56:39.923296 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":10158,"threads_started":1}
I20260924 00:56:39.925638 15192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:39.899957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:39347 (local address 127.0.0.1:37378)
0924 00:56:39.901872 (+ 1915us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:39.901916 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:39.902190 (+ 274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:39.903078 (+ 888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:39.903088 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:39.904304 (+ 1216us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:39.904758 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:39.904778 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:39.906544 (+ 1766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:39.906552 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:39.908516 (+ 1964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:39.908536 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:39.910059 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:39.910067 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:39.910357 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:39.911110 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:39.911150 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:39.914589 (+ 3439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:39.919746 (+ 5157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:39.919766 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:39.919778 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:39.920473 (+ 695us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:39.920988 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:39.921007 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:39.921014 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:39.921252 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:39.922222 (+ 970us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:39.922235 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:39.922511 (+ 276us) client_negotiation.cc:769] Sending connection context
0924 00:56:39.924579 (+ 2068us) client_negotiation.cc:240] Negotiation successful
0924 00:56:39.925013 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1655,"thread_start_us":1480,"threads_started":1}
I20260924 00:56:39.928474 14637 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.8.64.189:39347: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "1d362bb4f4dd41adaee4592166347b34"
member_type: NON_VOTER
last_known_addr {
host: "127.8.64.189"
port: 39347
}
attrs {
promote: true
}
}
dest_uuid: "db882bdd18eb4243928039daa8c49303"
cas_config_opid_index: -1
I20260924 00:56:39.929632 14637 consensus_queue.cc:237] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } peers { permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: NON_VOTER last_known_addr { host: "127.8.64.189" port: 39347 } attrs { promote: true } }
I20260924 00:56:39.948601 15141 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:39.947711 15196 raft_consensus.cc:2932] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 1d362bb4f4dd41adaee4592166347b34 (127.8.64.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } peers { permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: NON_VOTER last_known_addr { host: "127.8.64.189" port: 39347 } attrs { promote: true } } }
WARNING: no policy specified for oryx/127.8.64.188@KRBTEST.COM; defaulting to no policy
I20260924 00:56:39.952665 15141 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [term 1 FOLLOWER]: Refusing update from remote peer db882bdd18eb4243928039daa8c49303: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260924 00:56:39.953909 15197 consensus_queue.cc:1035] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: NON_VOTER last_known_addr { host: "127.8.64.189" port: 39347 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.64.188@KRBTEST.COM" created.
I20260924 00:56:39.960675 15196 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.8.64.189:39347
I20260924 00:56:39.961699 15099 master.cc:537] Master@127.8.64.189:39347 shutting down...
W20260924 00:56:39.967131 14624 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260924 00:56:39.966830 15198 sys_catalog.cc:455] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "db882bdd18eb4243928039daa8c49303" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } peers { permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: NON_VOTER last_known_addr { host: "127.8.64.189" port: 39347 } attrs { promote: true } } }
I20260924 00:56:39.967659 15198 sys_catalog.cc:458] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [sys.catalog]: This master's current role is: LEADER
W20260924 00:56:39.978571 14624 consensus_peers.cc:487] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 -> Peer 1d362bb4f4dd41adaee4592166347b34 (127.8.64.189:39347): Couldn't send request to peer 1d362bb4f4dd41adaee4592166347b34. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260924 00:56:39.978773 15099 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:56:39.979280 15099 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:56:39.979498 15099 tablet_replica.cc:330] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.64.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.188.keytab.
Entry for principal oryx/127.8.64.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.188.keytab.
I20260924 00:56:40.007026 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.188:36561
--webserver_interface=127.8.64.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.188.keytab
--principal=oryx/127.8.64.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:41079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:44743,127.8.64.189:39347,127.8.64.188:36561
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 00:56:40.082100 15099 master.cc:552] Master@127.8.64.189:39347 shutdown complete.
I20260924 00:56:40.082226 15099 master_runner.cc:305] Clearing existing system tablet
I20260924 00:56:40.082599 15099 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:40.082721 15099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:40.084964 15099 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260924 00:56:40.086436 15206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:40.086899 15099 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:40.087306 15099 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260924 00:56:40.087452 15099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-1/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-1/wal
uuid: "1d362bb4f4dd41adaee4592166347b34"
format_stamp: "Formatted at 2026-09-24 00:56:39 on dist-test-slave-mmhs"
server_key: "66eddb1f71e03689da3792060cdd6697"
server_key_iv: "1b93944cfc81500e431865d72f272cfd"
server_key_version: "encryptionkey@0"
I20260924 00:56:40.090947 15099 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:56:40.095497 15099 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260924 00:56:40.095688 15099 log.cc:1196] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260924 00:56:40.096077 15099 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34: Deleting consensus metadata
I20260924 00:56:40.096336 15099 master_runner.cc:315] Copying system tablet from 127.8.64.190:44743
I20260924 00:56:40.098910 15099 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34: tablet copy: Beginning tablet copy session from remote peer at address 127.8.64.190:44743
I20260924 00:56:40.111407 15170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:40.099649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39630 (local address 127.8.64.190:44743)
0924 00:56:40.099848 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:40.099857 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:40.100369 (+ 512us) server_negotiation.cc:407] Connection header received
0924 00:56:40.100615 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:40.100620 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:40.100692 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:40.100796 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:40.101605 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:40.102449 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:40.103746 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:40.104173 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:40.104302 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:40.106199 (+ 1897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:40.106205 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:40.106216 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:40.106246 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:40.108867 (+ 2621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:40.109457 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:40.109466 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:40.109470 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:40.109590 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:40.109956 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:40.109963 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:40.109967 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:40.110239 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:40.110428 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:40.110809 (+ 381us) server_negotiation.cc:299] Negotiation successful
0924 00:56:40.111012 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260924 00:56:40.111467 15212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:40.099634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44743 (local address 127.0.0.1:39630)
0924 00:56:40.100258 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:40.100278 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:40.100439 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:40.100820 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:40.100827 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:40.101261 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:40.101467 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:40.101475 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:40.102590 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:40.102596 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:40.103590 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:40.103600 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:40.104315 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:40.104320 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:40.104503 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:40.105140 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:40.105157 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:40.106016 (+ 859us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:40.108992 (+ 2976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:40.108999 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:40.109003 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:40.109343 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:40.109696 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:40.109701 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:40.109705 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:40.109853 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:40.110384 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:40.110389 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:40.110543 (+ 154us) client_negotiation.cc:769] Sending connection context
0924 00:56:40.110706 (+ 163us) client_negotiation.cc:240] Negotiation successful
0924 00:56:40.111029 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":354,"threads_started":1}
I20260924 00:56:40.112480 14657 tablet_copy_service.cc:140] P db882bdd18eb4243928039daa8c49303: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 1d362bb4f4dd41adaee4592166347b34 ({username='oryx', principal='oryx/127.8.64.189@KRBTEST.COM'} at 127.0.0.1:39630)
I20260924 00:56:40.112730 14657 tablet_copy_service.cc:161] P db882bdd18eb4243928039daa8c49303: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 1d362bb4f4dd41adaee4592166347b34 at {username='oryx', principal='oryx/127.8.64.189@KRBTEST.COM'} at 127.0.0.1:39630: session id = 1d362bb4f4dd41adaee4592166347b34-00000000000000000000000000000000
I20260924 00:56:40.115744 14657 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 00:56:40.117550 15099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260924 00:56:40.121932 15099 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34: tablet copy: Starting download of 0 data blocks...
I20260924 00:56:40.122223 15099 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34: tablet copy: Starting download of 1 WAL segments...
I20260924 00:56:40.124521 15099 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 00:56:40.127646 14657 tablet_copy_service.cc:342] P db882bdd18eb4243928039daa8c49303: Request end of tablet copy session 1d362bb4f4dd41adaee4592166347b34-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.8.64.189@KRBTEST.COM'} at 127.0.0.1:39630
I20260924 00:56:40.127825 14657 tablet_copy_service.cc:433] P db882bdd18eb4243928039daa8c49303: ending tablet copy session 1d362bb4f4dd41adaee4592166347b34-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 1d362bb4f4dd41adaee4592166347b34
I20260924 00:56:40.140599 15099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:40.147027 15214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:40.150705 15213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:40.154626 15099 server_base.cc:1034] running on GCE node
W20260924 00:56:40.154829 15216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:40.155236 15099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:40.155953 15099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:40.157140 15099 hybrid_clock.cc:648] HybridClock initialized: now 1790211400157067 us; error 49 us; skew 500 ppm
I20260924 00:56:40.157459 15099 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:40 dist-test-slave-mmhs krb5kdc[14592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211400, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.64.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:40.159447 15099 init.cc:377] Logged in from keytab as oryx/127.8.64.189@KRBTEST.COM (short username oryx)
I20260924 00:56:40.160780 15099 webserver.cc:460] Webserver started at http://127.8.64.189:42295/ using document root <none> and password file <none>
I20260924 00:56:40.161094 15099 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:40.161193 15099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:40.163995 15099 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260924 00:56:40.165573 15223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:40.166039 15099 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:56:40.166186 15099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-1/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-1/wal
uuid: "1d362bb4f4dd41adaee4592166347b34"
format_stamp: "Formatted at 2026-09-24 00:56:39 on dist-test-slave-mmhs"
server_key: "66eddb1f71e03689da3792060cdd6697"
server_key_iv: "1b93944cfc81500e431865d72f272cfd"
server_key_version: "encryptionkey@0"
I20260924 00:56:40.166371 15099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:40.203070 15099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:40.206565 15099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:40.248687 15099 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.189:39347
I20260924 00:56:40.248706 15274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.189:39347 every 8 connection(s)
I20260924 00:56:40.249696 15099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-1/data/info.pb
I20260924 00:56:40.256745 15094 heartbeater.cc:503] Master 127.8.64.190:44743 was elected leader, sending a full tablet report...
I20260924 00:56:40.258908 15275 sys_catalog.cc:263] Verifying existing consensus state
I20260924 00:56:40.262648 15275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34: Bootstrap starting.
I20260924 00:56:40.281384 15275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 00:56:40.281906 15275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34: Bootstrap complete.
I20260924 00:56:40.283010 15275 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } peers { permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: NON_VOTER last_known_addr { host: "127.8.64.189" port: 39347 } attrs { promote: true } }
I20260924 00:56:40.283376 15275 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1d362bb4f4dd41adaee4592166347b34, State: Initialized, Role: LEARNER
I20260924 00:56:40.283656 15275 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } peers { permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: NON_VOTER last_known_addr { host: "127.8.64.189" port: 39347 } attrs { promote: true } }
I20260924 00:56:40.284471 15278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } peers { permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: NON_VOTER last_known_addr { host: "127.8.64.189" port: 39347 } attrs { promote: true } } }
I20260924 00:56:40.284787 15275 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:56:40.284963 15278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [sys.catalog]: This master's current role is: LEARNER
I20260924 00:56:40.295625 15289 catalog_manager.cc:1259] Loaded cluster ID: d3b5b249561e46889e63beb4a75853a8
I20260924 00:56:40.295745 15289 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34: loading cluster ID for follower catalog manager: success
I20260924 00:56:40.299286 15289 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34: acquiring CA information for follower catalog manager: success
I20260924 00:56:40.301751 15289 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260924 00:56:40.417313 15204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:40.417632 15204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:40.417806 15204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:40.426929 15204 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:56:40.427073 15204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:40.427181 15204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:40.427285 15204 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:56:40.427381 15204 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:56:40.428215 15293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:40.421615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37384 (local address 127.8.64.189:39347)
0924 00:56:40.422263 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:40.422268 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:40.422282 (+ 14us) server_negotiation.cc:407] Connection header received
0924 00:56:40.422394 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:40.422398 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:40.422463 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:40.422558 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:56:40.423068 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:40.424050 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:40.426355 (+ 2305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:40.427248 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:40.427406 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:40.427632 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:40.427864 (+ 232us) server_negotiation.cc:299] Negotiation successful
0924 00:56:40.427955 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":401,"threads_started":1}
I20260924 00:56:40.428231 15192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:40.421436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:39347 (local address 127.0.0.1:37384)
0924 00:56:40.421615 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:40.421637 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:40.421792 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:40.422638 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:40.422644 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:40.422678 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:56:40.422899 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:40.422909 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:40.424200 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:40.424206 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:40.426144 (+ 1938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:40.426159 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:40.427463 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:40.427469 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:40.427685 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:40.427743 (+ 58us) client_negotiation.cc:769] Sending connection context
0924 00:56:40.427842 (+ 99us) client_negotiation.cc:240] Negotiation successful
0924 00:56:40.427975 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260924 00:56:40.429253 15250 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260924 00:56:40.429841 15278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db882bdd18eb4243928039daa8c49303. Latest consensus state: current_term: 1 leader_uuid: "db882bdd18eb4243928039daa8c49303" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } peers { permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: NON_VOTER last_known_addr { host: "127.8.64.189" port: 39347 } attrs { promote: true } } }
I20260924 00:56:40.430130 15278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [sys.catalog]: This master's current role is: LEARNER
I20260924 00:56:40.437930 15204 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:41079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:44743,127.8.64.189:39347,127.8.64.188:36561
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.188.keytab
--principal=oryx/127.8.64.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.188:36561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.64.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:40.439311 15204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:40.440587 15204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:40.452844 15295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:40.452940 15296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:40.452911 15298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:40.453873 15204 server_base.cc:1034] running on GCE node
I20260924 00:56:40.454668 15204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:40.456014 15204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:40.457231 15204 hybrid_clock.cc:648] HybridClock initialized: now 1790211400457186 us; error 60 us; skew 500 ppm
I20260924 00:56:40.457563 15204 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:40 dist-test-slave-mmhs krb5kdc[14592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211400, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.64.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:40.460805 15204 init.cc:377] Logged in from keytab as oryx/127.8.64.188@KRBTEST.COM (short username oryx)
I20260924 00:56:40.462570 15204 webserver.cc:460] Webserver started at http://127.8.64.188:42761/ using document root <none> and password file <none>
I20260924 00:56:40.463325 15204 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:40.463471 15204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:40.463770 15204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:56:40.467087 15204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-2/data/instance:
uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86"
format_stamp: "Formatted at 2026-09-24 00:56:40 on dist-test-slave-mmhs"
server_key: "e36e9a6c432079e8a7079027918c0826"
server_key_iv: "124f4004c0d9679a9d00d1f83b626465"
server_key_version: "encryptionkey@0"
I20260924 00:56:40.467900 15204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-2/wal/instance:
uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86"
format_stamp: "Formatted at 2026-09-24 00:56:40 on dist-test-slave-mmhs"
server_key: "e36e9a6c432079e8a7079027918c0826"
server_key_iv: "124f4004c0d9679a9d00d1f83b626465"
server_key_version: "encryptionkey@0"
I20260924 00:56:40.473140 15204 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260924 00:56:40.476461 15305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:40.477761 15204 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260924 00:56:40.477960 15204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-2/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-2/wal
uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86"
format_stamp: "Formatted at 2026-09-24 00:56:40 on dist-test-slave-mmhs"
server_key: "e36e9a6c432079e8a7079027918c0826"
server_key_iv: "124f4004c0d9679a9d00d1f83b626465"
server_key_version: "encryptionkey@0"
I20260924 00:56:40.478147 15204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:40.498277 15204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:40.502858 15204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:40.503136 15204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:40.523092 15204 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.188:36561
I20260924 00:56:40.523109 15356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.188:36561 every 8 connection(s)
I20260924 00:56:40.524433 15204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-2/data/info.pb
I20260924 00:56:40.528195 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 15204
I20260924 00:56:40.528223 15357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:40.537180 15357 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 00:56:40 dist-test-slave-mmhs krb5kdc[14592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.64.188@KRBTEST.COM
Sep 24 00:56:40 dist-test-slave-mmhs krb5kdc[14592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211400, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.64.188@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM
I20260924 00:56:40.551602 15360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:40.530435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43670 (local address 127.8.64.188:36561)
0924 00:56:40.531289 (+ 854us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:40.531307 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:40.531371 (+ 64us) server_negotiation.cc:407] Connection header received
0924 00:56:40.533191 (+ 1820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:40.533211 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:40.533612 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:40.534010 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:40.535468 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:40.536710 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:40.537898 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:40.538635 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:40.538818 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:40.543515 (+ 4697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:40.543523 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:40.543533 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:40.543564 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:40.546918 (+ 3354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:40.547675 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:40.547683 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:40.547691 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:40.547896 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:40.548284 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:40.548290 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:40.548293 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:40.549324 (+ 1031us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:40.549528 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:40.550067 (+ 539us) server_negotiation.cc:299] Negotiation successful
0924 00:56:40.550463 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":371,"threads_started":1}
I20260924 00:56:40.554687 15361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:40.539853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44743 (local address 127.0.0.1:39636)
0924 00:56:40.540460 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:40.540499 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:40.540650 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:40.541082 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:40.541087 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:40.541611 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:40.541828 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:40.541863 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:40.543122 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:40.543128 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:40.544295 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:40.544306 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:40.545263 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:40.545271 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:40.545489 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:40.546076 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:40.546095 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:40.548970 (+ 2875us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:40.552092 (+ 3122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:40.552100 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:40.552106 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:40.552642 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:40.552992 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:40.553000 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:40.553005 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:40.553121 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:40.553715 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:40.553720 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:40.553854 (+ 134us) client_negotiation.cc:769] Sending connection context
0924 00:56:40.554017 (+ 163us) client_negotiation.cc:240] Negotiation successful
0924 00:56:40.554225 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":345,"threads_started":1}
I20260924 00:56:40.554704 15170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:40.540001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39636 (local address 127.8.64.190:44743)
0924 00:56:40.540191 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:40.540199 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:40.540608 (+ 409us) server_negotiation.cc:407] Connection header received
0924 00:56:40.540779 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:40.540785 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:40.540873 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:40.541012 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:40.541999 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:40.542977 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:40.544437 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:40.545081 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:40.545203 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:40.549157 (+ 3954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:40.549162 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:40.549165 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:40.549197 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:40.551975 (+ 2778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:40.552758 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:40.552761 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:40.552764 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:40.552887 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:40.553249 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:40.553253 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:40.553256 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:40.553591 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:40.553737 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:40.554135 (+ 398us) server_negotiation.cc:299] Negotiation successful
0924 00:56:40.554385 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260924 00:56:40.555905 15357 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.64.189" port: 39347 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 00:56:40 dist-test-slave-mmhs krb5kdc[14592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211400, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.64.188@KRBTEST.COM for oryx/127.8.64.189@KRBTEST.COM
I20260924 00:56:40.568626 15361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:40.556629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:39347 (local address 127.0.0.1:37396)
0924 00:56:40.556802 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:40.556821 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:40.556911 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:40.557173 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:40.557176 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:40.557458 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:40.557582 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:40.557588 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:40.558574 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:40.558577 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:40.559559 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:40.559570 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:40.560290 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:40.560293 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:40.560432 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:40.561016 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:40.561033 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:40.563284 (+ 2251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:40.566488 (+ 3204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:40.566493 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:40.566498 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:40.566909 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:40.567242 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:40.567246 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:40.567248 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:40.567374 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:40.567928 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:40.567931 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:40.568084 (+ 153us) client_negotiation.cc:769] Sending connection context
0924 00:56:40.568191 (+ 107us) client_negotiation.cc:240] Negotiation successful
0924 00:56:40.568412 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260924 00:56:40.568740 15293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:40.556724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37396 (local address 127.8.64.189:39347)
0924 00:56:40.556897 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:40.556902 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:40.556918 (+ 16us) server_negotiation.cc:407] Connection header received
0924 00:56:40.557005 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:40.557009 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:40.557066 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:40.557144 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:40.557696 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:40.558468 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:40.559720 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:40.560139 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:40.560277 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:40.563509 (+ 3232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:40.563515 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:40.563518 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:40.563550 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:40.566285 (+ 2735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:40.567021 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:40.567024 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:40.567026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:40.567150 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:40.567488 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:40.567495 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:40.567498 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:40.567816 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:40.567934 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:40.568282 (+ 348us) server_negotiation.cc:299] Negotiation successful
0924 00:56:40.568477 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260924 00:56:40.569761 15357 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36561 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 00:56:40 dist-test-slave-mmhs krb5kdc[14592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211400, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.64.188@KRBTEST.COM for oryx/127.8.64.188@KRBTEST.COM
I20260924 00:56:40.582409 15360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:40.570645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43672 (local address 127.8.64.188:36561)
0924 00:56:40.570795 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:40.570801 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:40.570817 (+ 16us) server_negotiation.cc:407] Connection header received
0924 00:56:40.570956 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:40.570961 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:40.571057 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:40.571156 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:40.571924 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:40.573161 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:40.574209 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:40.574674 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:40.574816 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:40.577704 (+ 2888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:40.577710 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:40.577714 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:40.577750 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:40.579975 (+ 2225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:40.580552 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:40.580559 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:40.580563 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:40.580674 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:40.581032 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:40.581040 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:40.581043 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:40.581291 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:40.581430 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:40.581744 (+ 314us) server_negotiation.cc:299] Negotiation successful
0924 00:56:40.582074 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":169,"server-negotiator.queue_time_us":45}
I20260924 00:56:40.582412 15361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:40.570430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:36561 (local address 127.0.0.1:43672)
0924 00:56:40.570723 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:40.570740 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:40.570836 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:40.571170 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:40.571174 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:40.571535 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:40.571778 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:40.571788 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:40.573271 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:40.573285 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:40.574078 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:40.574088 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:40.574824 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:40.574829 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:40.575083 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:40.575597 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:40.575613 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:40.577542 (+ 1929us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:40.580101 (+ 2559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:40.580108 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:40.580112 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:40.580444 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:40.580783 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:40.580789 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:40.580793 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:40.580931 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:40.581399 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:40.581403 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:40.581545 (+ 142us) client_negotiation.cc:769] Sending connection context
0924 00:56:40.581677 (+ 132us) client_negotiation.cc:240] Negotiation successful
0924 00:56:40.581941 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"mutex_wait_us":22}
I20260924 00:56:40.587602 15357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86: Bootstrap starting.
I20260924 00:56:40.590927 15357 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:40.591995 15357 log.cc:824] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86: Log is configured to *not* fsync() on all Append() calls
I20260924 00:56:40.594894 15357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86: No bootstrap required, opened a new log
I20260924 00:56:40.601826 15357 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } peers { permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 39347 } } peers { permanent_uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36561 } }
I20260924 00:56:40.602167 15357 raft_consensus.cc:374] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:40.602329 15357 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ef84aecd3d45dd9caa57e3f2cc0d86, State: Initialized, Role: FOLLOWER
I20260924 00:56:40.602967 15357 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } peers { permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 39347 } } peers { permanent_uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36561 } }
I20260924 00:56:40.603783 15363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } peers { permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 39347 } } peers { permanent_uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36561 } } }
I20260924 00:56:40.604071 15363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:56:40.604800 15357 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86 [sys.catalog]: configured and running, proceeding with master startup.
W20260924 00:56:40.614126 15374 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 00:56:40.614261 15374 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 00:56:40.618958 15204 master_runner.cc:428] Detected that this master 52ef84aecd3d45dd9caa57e3f2cc0d86 is joining an existing cluster
I20260924 00:56:40.619118 15204 master_runner.cc:432] Initiating AddMaster RPC to add 127.8.64.188:36561
I20260924 00:56:40.644871 15293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:40.623288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37404 (local address 127.8.64.189:39347)
0924 00:56:40.623476 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:40.623482 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:40.624801 (+ 1319us) server_negotiation.cc:407] Connection header received
0924 00:56:40.625024 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:40.625029 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:40.625109 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:40.625182 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:40.627957 (+ 2775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:40.628990 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:40.631111 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:40.631714 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:40.631818 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:40.634371 (+ 2553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:40.634379 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:40.634385 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:40.634420 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:40.638058 (+ 3638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:40.640748 (+ 2690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:40.640753 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:40.640757 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:40.640919 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:40.641902 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:40.641908 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:40.641912 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:40.642219 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:40.642368 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:40.643838 (+ 1470us) server_negotiation.cc:299] Negotiation successful
0924 00:56:40.644087 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260924 00:56:40.646620 15360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:40.624871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43686 (local address 127.8.64.188:36561)
0924 00:56:40.625037 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:40.625042 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:40.626648 (+ 1606us) server_negotiation.cc:407] Connection header received
0924 00:56:40.626791 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:40.626797 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:40.626880 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:40.626997 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:40.628387 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:40.629551 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:40.631587 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:40.632381 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:40.632475 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:40.635388 (+ 2913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:40.635394 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:40.635398 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:40.635433 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:40.639513 (+ 4080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:40.641370 (+ 1857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:40.641375 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:40.641378 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:40.641535 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:40.642834 (+ 1299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:40.642839 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:40.642846 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:40.643169 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:40.643294 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:40.645996 (+ 2702us) server_negotiation.cc:299] Negotiation successful
0924 00:56:40.646210 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":53,"spinlock_wait_cycles":45312}
I20260924 00:56:40.647646 15382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:40.625153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:36561 (local address 127.0.0.1:43686)
0924 00:56:40.626518 (+ 1365us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:40.626543 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:40.626675 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:40.627145 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:40.627150 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:40.627859 (+ 709us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:40.628230 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:40.628242 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:40.629768 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:40.629773 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:40.631426 (+ 1653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:40.631441 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:40.633125 (+ 1684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:40.633131 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:40.633318 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:40.633959 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:40.634006 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:40.635206 (+ 1200us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:40.640096 (+ 4890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:40.640101 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:40.640105 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:40.640619 (+ 514us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:40.641657 (+ 1038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:40.641662 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:40.641665 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:40.641798 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:40.643353 (+ 1555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:40.643357 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:40.643544 (+ 187us) client_negotiation.cc:769] Sending connection context
0924 00:56:40.646912 (+ 3368us) client_negotiation.cc:240] Negotiation successful
0924 00:56:40.647196 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1240,"spinlock_wait_cycles":28416,"thread_start_us":1072,"threads_started":1}
I20260924 00:56:40.648041 15381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:40.623830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:39347 (local address 127.0.0.1:37404)
0924 00:56:40.624704 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:40.624721 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:40.624884 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:40.625422 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:40.625428 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:40.627445 (+ 2017us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:40.627797 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:40.627808 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:40.629185 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:40.629189 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:40.630943 (+ 1754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:40.630961 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:40.632109 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:40.632115 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:40.632310 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:40.632982 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:40.633006 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:40.634150 (+ 1144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:40.638204 (+ 4054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:40.638209 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:40.638213 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:40.639980 (+ 1767us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:40.641065 (+ 1085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:40.641070 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:40.641074 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:40.641253 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:40.642444 (+ 1191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:40.642449 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:40.642642 (+ 193us) client_negotiation.cc:769] Sending connection context
0924 00:56:40.645252 (+ 2610us) client_negotiation.cc:240] Negotiation successful
0924 00:56:40.645573 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":745,"thread_start_us":553,"threads_started":1}
I20260924 00:56:40.651785 15380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:40.623220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44743 (local address 127.0.0.1:39646)
0924 00:56:40.624229 (+ 1009us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:40.624255 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:40.624484 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:40.625032 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:40.625036 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:40.626271 (+ 1235us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:40.626709 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:40.626721 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:40.630369 (+ 3648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:40.630375 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:40.638382 (+ 8007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:40.638396 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:40.639630 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:40.639636 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:40.639820 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:40.640487 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:40.640511 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:40.641646 (+ 1135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:40.645923 (+ 4277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:40.645929 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:40.645933 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:40.647559 (+ 1626us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:40.648034 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:40.648039 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:40.648043 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:40.648229 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:40.650708 (+ 2479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:40.650714 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:40.650941 (+ 227us) client_negotiation.cc:769] Sending connection context
0924 00:56:40.651072 (+ 131us) client_negotiation.cc:240] Negotiation successful
0924 00:56:40.651367 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":717,"mutex_wait_us":6287,"thread_start_us":516,"threads_started":1}
I20260924 00:56:40.651938 15170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:40.623143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39646 (local address 127.8.64.190:44743)
0924 00:56:40.623370 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:40.623379 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:40.624352 (+ 973us) server_negotiation.cc:407] Connection header received
0924 00:56:40.624654 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:40.624661 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:40.624746 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:40.624819 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:40.626834 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:40.628002 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:40.638544 (+ 10542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:40.639232 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:40.639364 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:40.641892 (+ 2528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:40.641900 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:40.641907 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:40.641947 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:40.645747 (+ 3800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:40.647718 (+ 1971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:40.647724 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:40.647728 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:40.647888 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:40.650163 (+ 2275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:40.650169 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:40.650173 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:40.650576 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:40.650734 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:40.651281 (+ 547us) server_negotiation.cc:299] Negotiation successful
0924 00:56:40.651515 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260924 00:56:40.657779 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 14698
Sep 24 00:56:40 dist-test-slave-mmhs krb5kdc[14592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211397, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.64.190@KRBTEST.COM for oryx/127.8.64.188@KRBTEST.COM
Sep 24 00:56:40 dist-test-slave-mmhs krb5kdc[14592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.64.129@KRBTEST.COM
I20260924 00:56:40.675243 15360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:40.656900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43692 (local address 127.8.64.188:36561)
0924 00:56:40.657068 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:40.657073 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:40.657090 (+ 17us) server_negotiation.cc:407] Connection header received
0924 00:56:40.657193 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:40.657199 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:40.657280 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:40.658331 (+ 1051us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:40.658537 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:40.659584 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:40.661311 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:40.661962 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:40.662091 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:40.667612 (+ 5521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:40.667618 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:40.667625 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:40.667670 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:40.671791 (+ 4121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:40.672535 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:40.672540 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:40.672543 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:40.672681 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:40.673139 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:40.673144 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:40.673148 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:40.673515 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:40.673657 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:40.674239 (+ 582us) server_negotiation.cc:299] Negotiation successful
0924 00:56:40.674534 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260924 00:56:40.675271 15192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:40.656663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:36561 (local address 127.0.0.1:43692)
0924 00:56:40.656915 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:40.656936 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:40.657032 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:40.657444 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:40.657449 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:40.657910 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:40.658121 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:40.658132 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:40.659724 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:40.659730 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:40.661131 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:40.661145 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:40.662311 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:40.662317 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:40.662483 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:40.664348 (+ 1865us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:40.664370 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:40.667127 (+ 2757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:40.671954 (+ 4827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:40.671960 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:40.671964 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:40.672410 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:40.672814 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:40.672819 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:40.672823 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:40.672998 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:40.673719 (+ 721us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:40.673724 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:40.673903 (+ 179us) client_negotiation.cc:769] Sending connection context
0924 00:56:40.674103 (+ 200us) client_negotiation.cc:240] Negotiation successful
0924 00:56:40.674406 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122}
I20260924 00:56:40.676985 14637 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.8.64.188:36561: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86"
member_type: NON_VOTER
last_known_addr {
host: "127.8.64.188"
port: 36561
}
attrs {
promote: true
}
}
dest_uuid: "db882bdd18eb4243928039daa8c49303"
cas_config_opid_index: 5
I20260924 00:56:40.678017 14637 consensus_queue.cc:237] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } peers { permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: NON_VOTER last_known_addr { host: "127.8.64.189" port: 39347 } attrs { promote: true } } peers { permanent_uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86" member_type: NON_VOTER last_known_addr { host: "127.8.64.188" port: 36561 } attrs { promote: true } }
I20260924 00:56:40.680135 15198 raft_consensus.cc:2932] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 52ef84aecd3d45dd9caa57e3f2cc0d86 (127.8.64.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } peers { permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: NON_VOTER last_known_addr { host: "127.8.64.189" port: 39347 } attrs { promote: true } } peers { permanent_uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86" member_type: NON_VOTER last_known_addr { host: "127.8.64.188" port: 36561 } attrs { promote: true } } }
I20260924 00:56:40.680801 15332 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:40.681023 15383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:40.658856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54058 (local address 127.8.64.129:44385)
0924 00:56:40.659459 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:40.659468 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:40.659494 (+ 26us) server_negotiation.cc:407] Connection header received
0924 00:56:40.659559 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:40.659566 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:40.659845 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:40.660023 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:40.661278 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:40.662864 (+ 1586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:40.664242 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:40.665272 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:40.666002 (+ 730us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:40.672669 (+ 6667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:40.672684 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:40.672703 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:40.672750 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:40.676892 (+ 4142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:40.677707 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:40.677716 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:40.677720 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:40.677874 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:40.678379 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:40.678384 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:40.678388 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:40.678804 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:40.679113 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:40.680076 (+ 963us) server_negotiation.cc:299] Negotiation successful
0924 00:56:40.680357 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":296,"threads_started":1}
I20260924 00:56:40.682647 14821 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:56:40.684059 15198 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.8.64.188:36561
I20260924 00:56:40.684657 15204 master.cc:537] Master@127.8.64.188:36561 shutting down...
I20260924 00:56:40.684804 15332 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86 [term 1 FOLLOWER]: Refusing update from remote peer db882bdd18eb4243928039daa8c49303: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260924 00:56:40.686134 15250 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [term 1 LEARNER]: Refusing update from remote peer db882bdd18eb4243928039daa8c49303: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260924 00:56:40.686661 15198 sys_catalog.cc:455] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "db882bdd18eb4243928039daa8c49303" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } peers { permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: NON_VOTER last_known_addr { host: "127.8.64.189" port: 39347 } attrs { promote: true } } peers { permanent_uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86" member_type: NON_VOTER last_known_addr { host: "127.8.64.188" port: 36561 } attrs { promote: true } } }
I20260924 00:56:40.686959 15198 sys_catalog.cc:458] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [sys.catalog]: This master's current role is: LEADER
I20260924 00:56:40.693873 15384 consensus_queue.cc:1035] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: NON_VOTER last_known_addr { host: "127.8.64.189" port: 39347 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260924 00:56:40.694459 15384 consensus_queue.cc:1035] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86" member_type: NON_VOTER last_known_addr { host: "127.8.64.188" port: 36561 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:56:40.703639 15250 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 52ef84aecd3d45dd9caa57e3f2cc0d86 (127.8.64.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } peers { permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: NON_VOTER last_known_addr { host: "127.8.64.189" port: 39347 } attrs { promote: true } } peers { permanent_uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86" member_type: NON_VOTER last_known_addr { host: "127.8.64.188" port: 36561 } attrs { promote: true } } }
I20260924 00:56:40.708637 15278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "db882bdd18eb4243928039daa8c49303" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } peers { permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: NON_VOTER last_known_addr { host: "127.8.64.189" port: 39347 } attrs { promote: true } } peers { permanent_uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86" member_type: NON_VOTER last_known_addr { host: "127.8.64.188" port: 36561 } attrs { promote: true } } }
I20260924 00:56:40.708948 15278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [sys.catalog]: This master's current role is: LEARNER
I20260924 00:56:40.710196 15385 raft_consensus.cc:1050] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303: attempting to promote NON_VOTER 1d362bb4f4dd41adaee4592166347b34 to VOTER
I20260924 00:56:40.710966 15385 consensus_queue.cc:237] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } peers { permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 39347 } attrs { promote: false } } peers { permanent_uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86" member_type: NON_VOTER last_known_addr { host: "127.8.64.188" port: 36561 } attrs { promote: true } }
I20260924 00:56:40.714413 15250 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [term 1 LEARNER]: Refusing update from remote peer db882bdd18eb4243928039daa8c49303: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260924 00:56:40.715461 15384 consensus_queue.cc:1035] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 39347 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260924 00:56:40.717963 15384 raft_consensus.cc:2932] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 1d362bb4f4dd41adaee4592166347b34 (127.8.64.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } peers { permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 39347 } attrs { promote: false } } peers { permanent_uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86" member_type: NON_VOTER last_known_addr { host: "127.8.64.188" port: 36561 } attrs { promote: true } } }
I20260924 00:56:40.719779 15250 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 1d362bb4f4dd41adaee4592166347b34 (127.8.64.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } peers { permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 39347 } attrs { promote: false } } peers { permanent_uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86" member_type: NON_VOTER last_known_addr { host: "127.8.64.188" port: 36561 } attrs { promote: true } } }
I20260924 00:56:40.725757 15204 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:56:40.726267 15204 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:56:40.726536 15204 tablet_replica.cc:330] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86: stopping tablet replica
I20260924 00:56:40.729122 15278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "db882bdd18eb4243928039daa8c49303" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } peers { permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 39347 } attrs { promote: false } } peers { permanent_uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86" member_type: NON_VOTER last_known_addr { host: "127.8.64.188" port: 36561 } attrs { promote: true } } }
I20260924 00:56:40.729384 15278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:56:40.730453 15384 sys_catalog.cc:455] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "db882bdd18eb4243928039daa8c49303" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } peers { permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 39347 } attrs { promote: false } } peers { permanent_uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86" member_type: NON_VOTER last_known_addr { host: "127.8.64.188" port: 36561 } attrs { promote: true } } }
I20260924 00:56:40.730720 15384 sys_catalog.cc:458] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [sys.catalog]: This master's current role is: LEADER
I20260924 00:56:40.731967 15389 sys_catalog.cc:455] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "db882bdd18eb4243928039daa8c49303" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } peers { permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 39347 } attrs { promote: false } } peers { permanent_uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86" member_type: NON_VOTER last_known_addr { host: "127.8.64.188" port: 36561 } attrs { promote: true } } }
I20260924 00:56:40.732688 15192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:40.731905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:36561 (local address 127.0.0.1:43706)
0924 00:56:40.732311 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:40.732446 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:36561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259}
I20260924 00:56:40.732467 15389 sys_catalog.cc:458] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [sys.catalog]: This master's current role is: LEADER
W20260924 00:56:40.733682 14624 consensus_peers.cc:487] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 -> Peer 52ef84aecd3d45dd9caa57e3f2cc0d86 (127.8.64.188:36561): Couldn't send request to peer 52ef84aecd3d45dd9caa57e3f2cc0d86. Status: Network error: Client connection negotiation failed: client connection to 127.8.64.188:36561: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 00:56:40.735369 15204 master.cc:552] Master@127.8.64.188:36561 shutdown complete.
I20260924 00:56:40.735492 15204 master_runner.cc:305] Clearing existing system tablet
I20260924 00:56:40.736021 15204 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:40.736182 15204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:40.739706 15204 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260924 00:56:40.741431 15394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:40.741755 15204 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:40.742014 15204 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260924 00:56:40.742122 15204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-2/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-2/wal
uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86"
format_stamp: "Formatted at 2026-09-24 00:56:40 on dist-test-slave-mmhs"
server_key: "e36e9a6c432079e8a7079027918c0826"
server_key_iv: "124f4004c0d9679a9d00d1f83b626465"
server_key_version: "encryptionkey@0"
I20260924 00:56:40.747387 15204 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:56:40.755162 15204 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260924 00:56:40.755348 15204 log.cc:1196] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260924 00:56:40.755821 15204 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86: Deleting consensus metadata
I20260924 00:56:40.756114 15204 master_runner.cc:315] Copying system tablet from 127.8.64.190:44743
I20260924 00:56:40.758864 15204 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86: tablet copy: Beginning tablet copy session from remote peer at address 127.8.64.190:44743
I20260924 00:56:40.775324 15170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:40.760004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39660 (local address 127.8.64.190:44743)
0924 00:56:40.760204 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:40.760211 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:40.760280 (+ 69us) server_negotiation.cc:407] Connection header received
0924 00:56:40.760539 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:40.760545 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:40.760640 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:40.760755 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:40.762045 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:40.763268 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:40.765161 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:40.765806 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:40.765952 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:40.768400 (+ 2448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:40.768407 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:40.768412 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:40.768449 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:40.772277 (+ 3828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:40.773157 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:40.773163 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:40.773167 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:40.773321 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:40.773729 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:40.773743 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:40.773747 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:40.774105 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:40.774257 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:40.774737 (+ 480us) server_negotiation.cc:299] Negotiation successful
0924 00:56:40.774980 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260924 00:56:40.775497 15400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:40.759628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44743 (local address 127.0.0.1:39660)
0924 00:56:40.760162 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:40.760188 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:40.760394 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:40.760812 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:40.760819 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:40.761582 (+ 763us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:40.761854 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:40.761868 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:40.763438 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:40.763444 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:40.765005 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:40.765020 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:40.766025 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:40.766032 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:40.766231 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:40.767085 (+ 854us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:40.767112 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:40.768179 (+ 1067us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:40.772464 (+ 4285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:40.772471 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:40.772475 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:40.773012 (+ 537us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:40.773436 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:40.773442 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:40.773446 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:40.773616 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:40.774224 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:40.774229 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:40.774472 (+ 243us) client_negotiation.cc:769] Sending connection context
0924 00:56:40.774625 (+ 153us) client_negotiation.cc:240] Negotiation successful
0924 00:56:40.774963 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":596,"threads_started":1}
I20260924 00:56:40.776362 14657 tablet_copy_service.cc:140] P db882bdd18eb4243928039daa8c49303: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 52ef84aecd3d45dd9caa57e3f2cc0d86 ({username='oryx', principal='oryx/127.8.64.188@KRBTEST.COM'} at 127.0.0.1:39660)
I20260924 00:56:40.776540 14657 tablet_copy_service.cc:161] P db882bdd18eb4243928039daa8c49303: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 52ef84aecd3d45dd9caa57e3f2cc0d86 at {username='oryx', principal='oryx/127.8.64.188@KRBTEST.COM'} at 127.0.0.1:39660: session id = 52ef84aecd3d45dd9caa57e3f2cc0d86-00000000000000000000000000000000
I20260924 00:56:40.780253 14657 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 00:56:40.782198 15204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260924 00:56:40.789964 15204 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86: tablet copy: Starting download of 0 data blocks...
I20260924 00:56:40.790318 15204 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86: tablet copy: Starting download of 1 WAL segments...
I20260924 00:56:40.792923 15204 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 00:56:40.797971 14657 tablet_copy_service.cc:342] P db882bdd18eb4243928039daa8c49303: Request end of tablet copy session 52ef84aecd3d45dd9caa57e3f2cc0d86-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.8.64.188@KRBTEST.COM'} at 127.0.0.1:39660
I20260924 00:56:40.798130 14657 tablet_copy_service.cc:433] P db882bdd18eb4243928039daa8c49303: ending tablet copy session 52ef84aecd3d45dd9caa57e3f2cc0d86-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 52ef84aecd3d45dd9caa57e3f2cc0d86
I20260924 00:56:40.809508 15204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:40.812460 15401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:40.815611 15402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:40.829321 15404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:40.830627 15204 server_base.cc:1034] running on GCE node
I20260924 00:56:40.830998 15204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:40.831674 15204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:40.832989 15204 hybrid_clock.cc:648] HybridClock initialized: now 1790211400832915 us; error 42 us; skew 500 ppm
I20260924 00:56:40.833256 15204 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:40 dist-test-slave-mmhs krb5kdc[14592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211400, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.64.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:40.835479 15204 init.cc:377] Logged in from keytab as oryx/127.8.64.188@KRBTEST.COM (short username oryx)
I20260924 00:56:40.836892 15204 webserver.cc:460] Webserver started at http://127.8.64.188:43059/ using document root <none> and password file <none>
I20260924 00:56:40.837201 15204 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:40.837312 15204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:40.840415 15204 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
I20260924 00:56:40.842528 15411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:40.843231 15204 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260924 00:56:40.843377 15204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-2/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-2/wal
uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86"
format_stamp: "Formatted at 2026-09-24 00:56:40 on dist-test-slave-mmhs"
server_key: "e36e9a6c432079e8a7079027918c0826"
server_key_iv: "124f4004c0d9679a9d00d1f83b626465"
server_key_version: "encryptionkey@0"
I20260924 00:56:40.843611 15204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:40.845520 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 14698
I20260924 00:56:40.854481 15204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:40.859133 15204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:40.872323 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.129:44385
--local_ip_for_outbound_sockets=127.8.64.129
--tserver_master_addrs=127.8.64.190:44743
--webserver_port=37795
--webserver_interface=127.8.64.129
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.129.keytab
--principal=oryx/127.8.64.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:41079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.64.190:44743,127.8.64.189:39347
--tserver_master_addrs=127.8.64.190:44743,127.8.64.189:39347,127.8.64.188:36561 with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 00:56:40.902169 15204 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.188:36561
I20260924 00:56:40.902371 15463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.188:36561 every 8 connection(s)
I20260924 00:56:40.903216 15204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/master-2/data/info.pb
I20260924 00:56:40.909055 15464 sys_catalog.cc:263] Verifying existing consensus state
I20260924 00:56:40.913465 15464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86: Bootstrap starting.
I20260924 00:56:40.933096 15464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 00:56:40.933686 15464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86: Bootstrap complete.
I20260924 00:56:40.935073 15464 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } peers { permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 39347 } attrs { promote: false } } peers { permanent_uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86" member_type: NON_VOTER last_known_addr { host: "127.8.64.188" port: 36561 } attrs { promote: true } }
I20260924 00:56:40.935423 15464 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 52ef84aecd3d45dd9caa57e3f2cc0d86, State: Initialized, Role: LEARNER
I20260924 00:56:40.935727 15464 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } peers { permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 39347 } attrs { promote: false } } peers { permanent_uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86" member_type: NON_VOTER last_known_addr { host: "127.8.64.188" port: 36561 } attrs { promote: true } }
I20260924 00:56:40.936565 15467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } peers { permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 39347 } attrs { promote: false } } peers { permanent_uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86" member_type: NON_VOTER last_known_addr { host: "127.8.64.188" port: 36561 } attrs { promote: true } } }
I20260924 00:56:40.936919 15464 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:56:40.937048 15467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86 [sys.catalog]: This master's current role is: LEARNER
I20260924 00:56:40.946799 15478 catalog_manager.cc:1259] Loaded cluster ID: d3b5b249561e46889e63beb4a75853a8
I20260924 00:56:40.946902 15478 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86: loading cluster ID for follower catalog manager: success
I20260924 00:56:40.950416 15478 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86: acquiring CA information for follower catalog manager: success
I20260924 00:56:40.952889 15478 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 00:56:41.115134 15479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:41.106549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43716 (local address 127.8.64.188:36561)
0924 00:56:41.107212 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:41.107219 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:41.107239 (+ 20us) server_negotiation.cc:407] Connection header received
0924 00:56:41.107321 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:41.107326 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:41.107406 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:41.107529 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:56:41.108092 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:41.109800 (+ 1708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:41.112506 (+ 2706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:41.113927 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:41.114075 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:41.114349 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:41.114641 (+ 292us) server_negotiation.cc:299] Negotiation successful
0924 00:56:41.114779 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":354,"threads_started":1}
I20260924 00:56:41.115130 15192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:41.106253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:36561 (local address 127.0.0.1:43716)
0924 00:56:41.106543 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:41.106564 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:41.106691 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:41.107611 (+ 920us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:41.107618 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:41.107672 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:56:41.107889 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:41.107903 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:41.110012 (+ 2109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:41.110019 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:41.112288 (+ 2269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:41.112308 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:41.114153 (+ 1845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:41.114161 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:41.114441 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:41.114488 (+ 47us) client_negotiation.cc:769] Sending connection context
0924 00:56:41.114619 (+ 131us) client_negotiation.cc:240] Negotiation successful
0924 00:56:41.114783 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175}
I20260924 00:56:41.116516 15439 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260924 00:56:41.117161 15467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db882bdd18eb4243928039daa8c49303. Latest consensus state: current_term: 1 leader_uuid: "db882bdd18eb4243928039daa8c49303" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } peers { permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 39347 } attrs { promote: false } } peers { permanent_uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86" member_type: NON_VOTER last_known_addr { host: "127.8.64.188" port: 36561 } attrs { promote: true } } }
I20260924 00:56:41.117502 15467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86 [sys.catalog]: This master's current role is: LEARNER
W20260924 00:56:41.236766 15424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:41.237067 15424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:41.237309 15424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:41.244839 15424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:41.244987 15424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:41.245157 15424 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.129
I20260924 00:56:41.259599 15424 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:41079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.129.keytab
--principal=oryx/127.8.64.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.129:44385
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.64.129
--webserver_port=37795
--tserver_master_addrs=127.8.64.190:44743,127.8.64.189:39347,127.8.64.188:36561
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.129
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:41.261605 15424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:41.263274 15424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:41.278594 15485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:41.278502 15486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:41.279071 15488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:41.279630 15424 server_base.cc:1034] running on GCE node
I20260924 00:56:41.280367 15424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:41.281399 15424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:41.282634 15424 hybrid_clock.cc:648] HybridClock initialized: now 1790211401282574 us; error 29 us; skew 500 ppm
I20260924 00:56:41.282992 15424 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:41 dist-test-slave-mmhs krb5kdc[14592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211401, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.64.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:41.287051 15424 init.cc:377] Logged in from keytab as oryx/127.8.64.129@KRBTEST.COM (short username oryx)
I20260924 00:56:41.288636 15424 webserver.cc:460] Webserver started at http://127.8.64.129:37795/ using document root <none> and password file <none>
I20260924 00:56:41.289351 15424 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:41.289495 15424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:41.294837 15424 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260924 00:56:41.297452 15495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:41.298893 15424 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 00:56:41.299073 15424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-0/wal
uuid: "22c762dcd2e94b6f91721a9571613065"
format_stamp: "Formatted at 2026-09-24 00:56:38 on dist-test-slave-mmhs"
server_key: "dcda79cd347d800d081d192118aac951"
server_key_iv: "bf54d53d6921c835aa6f5e295b122cfd"
server_key_version: "encryptionkey@0"
I20260924 00:56:41.300022 15424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:41.317430 15424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:41.321704 15424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:41.322042 15424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:41.323035 15424 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:56:41.324298 15424 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:56:41.324421 15424 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:41.324558 15424 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:56:41.324645 15424 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:41.363077 15424 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.129:44385
I20260924 00:56:41.363127 15607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.129:44385 every 8 connection(s)
I20260924 00:56:41.364603 15424 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
I20260924 00:56:41.372491 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 15424
I20260924 00:56:41.372839 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 14832
Sep 24 00:56:41 dist-test-slave-mmhs krb5kdc[14592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211401, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.64.129@KRBTEST.COM for oryx/127.8.64.189@KRBTEST.COM
Sep 24 00:56:41 dist-test-slave-mmhs krb5kdc[14592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211401, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.64.129@KRBTEST.COM for oryx/127.8.64.188@KRBTEST.COM
Sep 24 00:56:41 dist-test-slave-mmhs krb5kdc[14592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211401, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.64.129@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM
I20260924 00:56:41.399578 15479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:41.368555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:59607 (local address 127.8.64.188:36561)
0924 00:56:41.368723 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:41.368729 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:41.370056 (+ 1327us) server_negotiation.cc:407] Connection header received
0924 00:56:41.372621 (+ 2565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:41.372627 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:41.372738 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:41.372836 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:41.376551 (+ 3715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:41.377994 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:41.381598 (+ 3604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:41.382547 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:41.382672 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:41.390787 (+ 8115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:41.390793 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:41.390797 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:41.390861 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:41.395098 (+ 4237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:41.396621 (+ 1523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:41.396626 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:41.396630 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:41.396793 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:41.397221 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:41.397229 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:41.397235 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:41.398405 (+ 1170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:41.398533 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:41.399041 (+ 508us) server_negotiation.cc:299] Negotiation successful
0924 00:56:41.399259 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260924 00:56:41.400061 15615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:41.370654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:39347 (local address 127.8.64.129:38755)
0924 00:56:41.371345 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:41.371369 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:41.372298 (+ 929us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:41.373195 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:41.373205 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:41.374814 (+ 1609us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:41.376279 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:41.376306 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:41.378762 (+ 2456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:41.378771 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:41.381372 (+ 2601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:41.381392 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:41.382772 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:41.382782 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:41.383206 (+ 424us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:41.384269 (+ 1063us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:41.384308 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:41.389624 (+ 5316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:41.395168 (+ 5544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:41.395177 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:41.395201 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:41.395941 (+ 740us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:41.396351 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:41.396356 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:41.396360 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:41.396620 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:41.397283 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:41.397293 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:41.397728 (+ 435us) client_negotiation.cc:769] Sending connection context
0924 00:56:41.398048 (+ 320us) client_negotiation.cc:240] Negotiation successful
0924 00:56:41.398608 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"mutex_wait_us":313,"spinlock_wait_cycles":1280,"thread_start_us":288,"threads_started":1}
I20260924 00:56:41.401177 15614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:41.368805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:36561 (local address 127.8.64.129:59607)
0924 00:56:41.369868 (+ 1063us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:41.369924 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:41.372308 (+ 2384us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:41.373193 (+ 885us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:41.373205 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:41.374788 (+ 1583us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:41.376275 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:41.376305 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:41.378224 (+ 1919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:41.378232 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:41.381385 (+ 3153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:41.381405 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:41.383047 (+ 1642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:41.383056 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:41.383745 (+ 689us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:41.386400 (+ 2655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:41.386433 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:41.390544 (+ 4111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:41.395261 (+ 4717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:41.395267 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:41.395271 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:41.396484 (+ 1213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:41.396945 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:41.396949 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:41.396952 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:41.397078 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:41.398628 (+ 1550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:41.398632 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:41.398761 (+ 129us) client_negotiation.cc:769] Sending connection context
0924 00:56:41.400100 (+ 1339us) client_negotiation.cc:240] Negotiation successful
0924 00:56:41.400642 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":910,"mutex_wait_us":731,"thread_start_us":453,"threads_started":1}
Sep 24 00:56:41 dist-test-slave-mmhs krb5kdc[14592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.64.130@KRBTEST.COM
I20260924 00:56:41.401963 15616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:41.370709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:38755 (local address 127.8.64.189:39347)
0924 00:56:41.371883 (+ 1174us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:41.371888 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:41.371909 (+ 21us) server_negotiation.cc:407] Connection header received
0924 00:56:41.372721 (+ 812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:41.372727 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:41.372802 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:41.373521 (+ 719us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:41.376525 (+ 3004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:41.378539 (+ 2014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:41.381581 (+ 3042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:41.382470 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:41.383372 (+ 902us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:41.390643 (+ 7271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:41.390649 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:41.390656 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:41.390701 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:41.395004 (+ 4303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:41.396064 (+ 1060us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:41.396072 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:41.396075 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:41.396217 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:41.396762 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:41.396767 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:41.396770 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:41.397142 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:41.401034 (+ 3892us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:41.401205 (+ 171us) server_negotiation.cc:299] Negotiation successful
0924 00:56:41.401568 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1055,"thread_start_us":946,"threads_started":1}
I20260924 00:56:41.403400 15609 heartbeater.cc:347] Connected to a master server at 127.8.64.188:36561
I20260924 00:56:41.403503 15610 heartbeater.cc:347] Connected to a master server at 127.8.64.189:39347
I20260924 00:56:41.403746 15610 heartbeater.cc:464] Registering TS with master...
I20260924 00:56:41.403766 15609 heartbeater.cc:464] Registering TS with master...
I20260924 00:56:41.404793 15609 heartbeater.cc:511] Master 127.8.64.188:36561 requested a full tablet report, sending...
I20260924 00:56:41.404817 15610 heartbeater.cc:511] Master 127.8.64.189:39347 requested a full tablet report, sending...
I20260924 00:56:41.406127 15612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:41.368045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:44045 (local address 127.8.64.190:44743)
0924 00:56:41.368526 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:41.368532 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:41.370048 (+ 1516us) server_negotiation.cc:407] Connection header received
0924 00:56:41.372810 (+ 2762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:41.372817 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:41.372915 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:41.372999 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:41.388292 (+ 15293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:41.389412 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:41.391314 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:41.392010 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:41.392143 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:41.397525 (+ 5382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:41.397532 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:41.397537 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:41.397576 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:41.401300 (+ 3724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:41.402087 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:41.402092 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:41.402115 (+ 23us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:41.402699 (+ 584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:41.403216 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:41.403224 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:41.403229 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:41.403614 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:41.403766 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:41.405532 (+ 1766us) server_negotiation.cc:299] Negotiation successful
0924 00:56:41.405757 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":242,"threads_started":1}
I20260924 00:56:41.406257 15613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:41.368791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44743 (local address 127.8.64.129:44045)
0924 00:56:41.369816 (+ 1025us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:41.369921 (+ 105us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:41.372656 (+ 2735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:41.373238 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:41.373246 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:41.375054 (+ 1808us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:41.388051 (+ 12997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:41.388074 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:41.389562 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:41.389565 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:41.391114 (+ 1549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:41.391131 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:41.393210 (+ 2079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:41.393220 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:41.393435 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:41.394482 (+ 1047us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:41.394508 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:41.397321 (+ 2813us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:41.401460 (+ 4139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:41.401466 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:41.401470 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:41.401935 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:41.402852 (+ 917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:41.402857 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:41.402860 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:41.403046 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:41.404739 (+ 1693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:41.404747 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:41.405227 (+ 480us) client_negotiation.cc:769] Sending connection context
0924 00:56:41.405380 (+ 153us) client_negotiation.cc:240] Negotiation successful
0924 00:56:41.405705 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":714,"mutex_wait_us":76,"spinlock_wait_cycles":190592,"thread_start_us":438,"threads_started":1}
I20260924 00:56:41.407269 15608 heartbeater.cc:347] Connected to a master server at 127.8.64.190:44743
I20260924 00:56:41.407410 15608 heartbeater.cc:464] Registering TS with master...
I20260924 00:56:41.407377 15240 ts_manager.cc:194] Registered new tserver with Master: 22c762dcd2e94b6f91721a9571613065 (127.8.64.129:44385)
I20260924 00:56:41.407802 15608 heartbeater.cc:511] Master 127.8.64.190:44743 requested a full tablet report, sending...
I20260924 00:56:41.409780 15429 ts_manager.cc:194] Registered new tserver with Master: 22c762dcd2e94b6f91721a9571613065 (127.8.64.129:44385)
I20260924 00:56:41.410742 14637 ts_manager.cc:194] Re-registered known tserver with Master: 22c762dcd2e94b6f91721a9571613065 (127.8.64.129:44385)
I20260924 00:56:41.411576 15619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:41.374181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50984 (local address 127.8.64.130:44619)
0924 00:56:41.389462 (+ 15281us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:41.389472 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:41.389505 (+ 33us) server_negotiation.cc:407] Connection header received
0924 00:56:41.389573 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:41.389582 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:41.389911 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:41.390076 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:41.392199 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:41.393632 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:41.395585 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:41.396478 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:41.396611 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:41.403854 (+ 7243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:41.403872 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:41.403888 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:41.403934 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:41.408333 (+ 4399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:41.409255 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:41.409261 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:41.409264 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:41.409476 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:41.409926 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:41.409930 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:41.409935 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:41.410340 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:41.410663 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:41.410968 (+ 305us) server_negotiation.cc:299] Negotiation successful
0924 00:56:41.411225 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15086,"thread_start_us":16101,"threads_started":1}
I20260924 00:56:41.411656 14637 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.64.129@KRBTEST.COM'} at 127.8.64.129:44045
I20260924 00:56:41.412863 14956 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:56:41.496030 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 14832
I20260924 00:56:41.513688 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.130:44619
--local_ip_for_outbound_sockets=127.8.64.130
--tserver_master_addrs=127.8.64.190:44743
--webserver_port=33341
--webserver_interface=127.8.64.130
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.130.keytab
--principal=oryx/127.8.64.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:41079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.64.190:44743,127.8.64.189:39347
--tserver_master_addrs=127.8.64.190:44743,127.8.64.189:39347,127.8.64.188:36561 with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 00:56:41.678740 15198 sys_catalog.cc:455] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "db882bdd18eb4243928039daa8c49303" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } peers { permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 39347 } attrs { promote: false } } peers { permanent_uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86" member_type: NON_VOTER last_known_addr { host: "127.8.64.188" port: 36561 } attrs { promote: true } } }
I20260924 00:56:41.679172 15198 sys_catalog.cc:458] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [sys.catalog]: This master's current role is: LEADER
I20260924 00:56:41.680380 15623 raft_consensus.cc:1050] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303: attempting to promote NON_VOTER 52ef84aecd3d45dd9caa57e3f2cc0d86 to VOTER
I20260924 00:56:41.681059 15623 consensus_queue.cc:237] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } peers { permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 39347 } attrs { promote: false } } peers { permanent_uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36561 } attrs { promote: false } }
I20260924 00:56:41.683118 15439 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86 [term 1 LEARNER]: Refusing update from remote peer db882bdd18eb4243928039daa8c49303: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260924 00:56:41.683251 15250 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [term 1 FOLLOWER]: Refusing update from remote peer db882bdd18eb4243928039daa8c49303: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260924 00:56:41.683987 15622 consensus_queue.cc:1035] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 39347 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260924 00:56:41.684407 15625 consensus_queue.cc:1035] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36561 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260924 00:56:41.686991 15622 raft_consensus.cc:2932] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 52ef84aecd3d45dd9caa57e3f2cc0d86 (127.8.64.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } peers { permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 39347 } attrs { promote: false } } peers { permanent_uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36561 } attrs { promote: false } } }
I20260924 00:56:41.687667 15250 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 52ef84aecd3d45dd9caa57e3f2cc0d86 (127.8.64.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } peers { permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 39347 } attrs { promote: false } } peers { permanent_uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36561 } attrs { promote: false } } }
I20260924 00:56:41.691218 15198 sys_catalog.cc:455] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "db882bdd18eb4243928039daa8c49303" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } peers { permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 39347 } attrs { promote: false } } peers { permanent_uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36561 } attrs { promote: false } } }
I20260924 00:56:41.691480 15198 sys_catalog.cc:458] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [sys.catalog]: This master's current role is: LEADER
I20260924 00:56:41.692183 15198 sys_catalog.cc:455] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "db882bdd18eb4243928039daa8c49303" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } peers { permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 39347 } attrs { promote: false } } peers { permanent_uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36561 } attrs { promote: false } } }
I20260924 00:56:41.692497 15198 sys_catalog.cc:458] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [sys.catalog]: This master's current role is: LEADER
I20260924 00:56:41.693096 15627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "db882bdd18eb4243928039daa8c49303" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } peers { permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 39347 } attrs { promote: false } } peers { permanent_uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36561 } attrs { promote: false } } }
I20260924 00:56:41.693507 15627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:56:41.695109 15198 sys_catalog.cc:455] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "db882bdd18eb4243928039daa8c49303" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } peers { permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 39347 } attrs { promote: false } } peers { permanent_uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36561 } attrs { promote: false } } }
I20260924 00:56:41.695343 15198 sys_catalog.cc:458] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303 [sys.catalog]: This master's current role is: LEADER
I20260924 00:56:41.695864 15439 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 52ef84aecd3d45dd9caa57e3f2cc0d86 (127.8.64.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } peers { permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 39347 } attrs { promote: false } } peers { permanent_uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36561 } attrs { promote: false } } }
I20260924 00:56:41.700153 15628 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "db882bdd18eb4243928039daa8c49303" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } peers { permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 39347 } attrs { promote: false } } peers { permanent_uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36561 } attrs { promote: false } } }
I20260924 00:56:41.700551 15628 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86 [sys.catalog]: This master's current role is: FOLLOWER
W20260924 00:56:41.847855 15621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:41.848173 15621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:41.848348 15621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:41.855935 15621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:41.856074 15621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:41.856261 15621 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.130
I20260924 00:56:41.865842 15621 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:41079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.130.keytab
--principal=oryx/127.8.64.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.130:44619
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.64.130
--webserver_port=33341
--tserver_master_addrs=127.8.64.190:44743,127.8.64.189:39347,127.8.64.188:36561
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.130
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:41.867343 15621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:41.868726 15621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:41.882661 15634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:41.882740 15636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:41.882913 15633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:41.883271 15621 server_base.cc:1034] running on GCE node
I20260924 00:56:41.883894 15621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:41.884855 15621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:41.886127 15621 hybrid_clock.cc:648] HybridClock initialized: now 1790211401886053 us; error 65 us; skew 500 ppm
I20260924 00:56:41.886512 15621 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:41 dist-test-slave-mmhs krb5kdc[14592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211401, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.64.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:41.889935 15621 init.cc:377] Logged in from keytab as oryx/127.8.64.130@KRBTEST.COM (short username oryx)
I20260924 00:56:41.891407 15621 webserver.cc:460] Webserver started at http://127.8.64.130:33341/ using document root <none> and password file <none>
I20260924 00:56:41.892053 15621 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:41.892165 15621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:41.897385 15621 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260924 00:56:41.899996 15643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:41.901356 15621 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260924 00:56:41.901520 15621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-1/wal
uuid: "72f23ce899264c60b7b9818e9cc2797c"
format_stamp: "Formatted at 2026-09-24 00:56:38 on dist-test-slave-mmhs"
server_key: "f2bc4483bcddb6bb9f658e8ffc69dd48"
server_key_iv: "20e951698e883a24b9a2bd3045048b04"
server_key_version: "encryptionkey@0"
I20260924 00:56:41.902549 15621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:41.921551 15621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:41.926033 15621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:41.926384 15621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:41.927496 15621 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:56:41.928941 15621 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:56:41.929064 15621 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:41.929196 15621 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:56:41.929284 15621 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:41.960819 15621 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.130:44619
I20260924 00:56:41.960918 15756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.130:44619 every 8 connection(s)
I20260924 00:56:41.962392 15621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
I20260924 00:56:41.965057 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 15621
I20260924 00:56:41.965361 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 14966
Sep 24 00:56:41 dist-test-slave-mmhs krb5kdc[14592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211401, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.64.130@KRBTEST.COM for oryx/127.8.64.188@KRBTEST.COM
Sep 24 00:56:41 dist-test-slave-mmhs krb5kdc[14592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.64.131@KRBTEST.COM
Sep 24 00:56:41 dist-test-slave-mmhs krb5kdc[14592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211401, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.64.130@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM
I20260924 00:56:42.005128 15765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:41.969185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:47505 (local address 127.8.64.188:36561)
0924 00:56:41.978888 (+ 9703us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:41.978898 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:41.978928 (+ 30us) server_negotiation.cc:407] Connection header received
0924 00:56:41.979017 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:41.979025 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:41.979139 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:41.983328 (+ 4189us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:41.983604 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:41.985873 (+ 2269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:41.988822 (+ 2949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:41.989537 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:41.989655 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:41.995355 (+ 5700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:41.995362 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:41.995367 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:41.995408 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:41.999095 (+ 3687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:42.000193 (+ 1098us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:42.000198 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:42.000201 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:42.000737 (+ 536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:42.002639 (+ 1902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:42.002648 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:42.002652 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:42.003026 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:42.003281 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:42.004388 (+ 1107us) server_negotiation.cc:299] Negotiation successful
0924 00:56:42.004663 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9519,"thread_start_us":9335,"threads_started":1}
Sep 24 00:56:42 dist-test-slave-mmhs krb5kdc[14592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211401, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.64.130@KRBTEST.COM for oryx/127.8.64.189@KRBTEST.COM
I20260924 00:56:42.008249 15762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:41.966198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:36561 (local address 127.8.64.130:47505)
0924 00:56:41.967442 (+ 1244us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:41.967499 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:41.970712 (+ 3213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:41.980566 (+ 9854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:41.980585 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:41.981795 (+ 1210us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:41.983065 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:41.983099 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:41.986105 (+ 3006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:41.986110 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:41.988621 (+ 2511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:41.988638 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:41.989952 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:41.989960 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:41.990311 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:41.991604 (+ 1293us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:41.991636 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:41.995097 (+ 3461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:41.999265 (+ 4168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:41.999274 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:41.999294 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:42.000034 (+ 740us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:42.002183 (+ 2149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:42.002190 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:42.002194 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:42.002470 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:42.003354 (+ 884us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:42.003364 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:42.003839 (+ 475us) client_negotiation.cc:769] Sending connection context
0924 00:56:42.005466 (+ 1627us) client_negotiation.cc:240] Negotiation successful
0924 00:56:42.006163 (+ 697us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":807,"thread_start_us":8140,"threads_started":1}
I20260924 00:56:42.008713 15764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:41.967075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44743 (local address 127.8.64.130:56323)
0924 00:56:41.979847 (+ 12772us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:41.979873 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:41.980014 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:41.991392 (+ 11378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:41.991400 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:41.992230 (+ 830us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:41.992613 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:41.992628 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:41.994460 (+ 1832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:41.994469 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:41.996373 (+ 1904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:41.996389 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:41.997527 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:41.997535 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:41.997734 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:41.998335 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:41.998362 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:42.001438 (+ 3076us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:42.005803 (+ 4365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:42.005808 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:42.005813 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:42.006485 (+ 672us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:42.006936 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:42.006940 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:42.006944 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:42.007093 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:42.007727 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:42.007734 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:42.007889 (+ 155us) client_negotiation.cc:769] Sending connection context
0924 00:56:42.008008 (+ 119us) client_negotiation.cc:240] Negotiation successful
0924 00:56:42.008319 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12601,"thread_start_us":12409,"threads_started":1}
I20260924 00:56:42.010032 15763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:41.967064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:56323 (local address 127.8.64.190:44743)
0924 00:56:41.991014 (+ 23950us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:41.991020 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:41.991040 (+ 20us) server_negotiation.cc:407] Connection header received
0924 00:56:41.991097 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:41.991103 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:41.991190 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:41.991306 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:41.993004 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:41.994244 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:41.996580 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:41.997305 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:41.998048 (+ 743us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:42.001660 (+ 3612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:42.001666 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:42.001671 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:42.001721 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:42.005644 (+ 3923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:42.006650 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:42.006657 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:42.006661 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:42.006811 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:42.007245 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:42.007252 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:42.007258 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:42.007603 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:42.009318 (+ 1715us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:42.009434 (+ 116us) server_negotiation.cc:299] Negotiation successful
0924 00:56:42.009658 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23821,"thread_start_us":23949,"threads_started":1}
I20260924 00:56:42.010182 15758 heartbeater.cc:347] Connected to a master server at 127.8.64.188:36561
I20260924 00:56:42.010478 15758 heartbeater.cc:464] Registering TS with master...
I20260924 00:56:42.011452 15758 heartbeater.cc:511] Master 127.8.64.188:36561 requested a full tablet report, sending...
I20260924 00:56:42.011617 15757 heartbeater.cc:347] Connected to a master server at 127.8.64.190:44743
I20260924 00:56:42.011819 15757 heartbeater.cc:464] Registering TS with master...
I20260924 00:56:42.012329 15757 heartbeater.cc:511] Master 127.8.64.190:44743 requested a full tablet report, sending...
I20260924 00:56:42.012451 15766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:41.969536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60776 (local address 127.8.64.131:33651)
0924 00:56:41.975792 (+ 6256us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:41.975802 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:41.985325 (+ 9523us) server_negotiation.cc:407] Connection header received
0924 00:56:41.985512 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:41.985569 (+ 57us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:41.985831 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:41.986019 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:41.988346 (+ 2327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:41.989954 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:41.991925 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:41.992763 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:41.992887 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:42.001708 (+ 8821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:42.001722 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:42.001736 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:42.001770 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:42.006035 (+ 4265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:42.007048 (+ 1013us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:42.007057 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:42.007061 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:42.007221 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:42.007685 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:42.007693 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:42.007699 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:42.008106 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:42.008367 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:42.009988 (+ 1621us) server_negotiation.cc:299] Negotiation successful
0924 00:56:42.010265 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6067,"thread_start_us":6643,"threads_started":1}
I20260924 00:56:42.013285 15429 ts_manager.cc:194] Registered new tserver with Master: 72f23ce899264c60b7b9818e9cc2797c (127.8.64.130:44619)
I20260924 00:56:42.013494 14637 ts_manager.cc:194] Re-registered known tserver with Master: 72f23ce899264c60b7b9818e9cc2797c (127.8.64.130:44619)
I20260924 00:56:42.013949 15089 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:56:42.014493 15768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:41.975831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:39347 (local address 127.8.64.130:54735)
0924 00:56:41.996407 (+ 20576us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:41.996427 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:41.996543 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:41.997284 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:41.997289 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:41.997893 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:41.998278 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:41.998310 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:41.999965 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:41.999968 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:42.001948 (+ 1980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:42.001961 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:42.003587 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:42.003593 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:42.004248 (+ 655us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:42.004803 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:42.004826 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:42.007549 (+ 2723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:42.011630 (+ 4081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:42.011636 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:42.011642 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:42.012140 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:42.012612 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:42.012617 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:42.012621 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:42.012794 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:42.013539 (+ 745us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:42.013547 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:42.013686 (+ 139us) client_negotiation.cc:769] Sending connection context
0924 00:56:42.013809 (+ 123us) client_negotiation.cc:240] Negotiation successful
0924 00:56:42.014069 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19711,"thread_start_us":20021,"threads_started":1}
I20260924 00:56:42.014813 14637 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.64.130@KRBTEST.COM'} at 127.8.64.130:56323
I20260924 00:56:42.015395 15767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:41.976018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:54735 (local address 127.8.64.189:39347)
0924 00:56:41.996003 (+ 19985us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:41.996010 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:41.996808 (+ 798us) server_negotiation.cc:407] Connection header received
0924 00:56:41.996888 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:41.996893 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:41.996993 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:41.997105 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:41.998477 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:41.999715 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:42.002108 (+ 2393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:42.003239 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:42.003371 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:42.007763 (+ 4392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:42.007770 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:42.007774 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:42.007837 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:42.011469 (+ 3632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:42.012290 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:42.012297 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:42.012300 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:42.012477 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:42.012926 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:42.012931 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:42.012934 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:42.013310 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:42.014743 (+ 1433us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:42.014878 (+ 135us) server_negotiation.cc:299] Negotiation successful
0924 00:56:42.015097 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19857,"thread_start_us":23794,"threads_started":1}
I20260924 00:56:42.016216 15759 heartbeater.cc:347] Connected to a master server at 127.8.64.189:39347
I20260924 00:56:42.016333 15759 heartbeater.cc:464] Registering TS with master...
I20260924 00:56:42.017066 15759 heartbeater.cc:511] Master 127.8.64.189:39347 requested a full tablet report, sending...
I20260924 00:56:42.018139 15240 ts_manager.cc:194] Registered new tserver with Master: 72f23ce899264c60b7b9818e9cc2797c (127.8.64.130:44619)
I20260924 00:56:42.097734 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 14966
I20260924 00:56:42.114498 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.131:33651
--local_ip_for_outbound_sockets=127.8.64.131
--tserver_master_addrs=127.8.64.190:44743
--webserver_port=37655
--webserver_interface=127.8.64.131
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.131.keytab
--principal=oryx/127.8.64.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:41079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.64.190:44743,127.8.64.189:39347
--tserver_master_addrs=127.8.64.190:44743,127.8.64.189:39347,127.8.64.188:36561 with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 00:56:42.414768 15608 heartbeater.cc:503] Master 127.8.64.190:44743 was elected leader, sending a full tablet report...
W20260924 00:56:42.443244 15770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:42.443516 15770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:42.443610 15770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:42.450920 15770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:42.451054 15770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:42.451177 15770 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.131
I20260924 00:56:42.460814 15770 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:41079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.131.keytab
--principal=oryx/127.8.64.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.131:33651
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.64.131
--webserver_port=37655
--tserver_master_addrs=127.8.64.190:44743,127.8.64.189:39347,127.8.64.188:36561
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.131
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:42.462070 15770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:42.463508 15770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:42.476832 15775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:42.476867 15776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:42.477207 15778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:42.478565 15770 server_base.cc:1034] running on GCE node
I20260924 00:56:42.479116 15770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:42.480034 15770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:42.481254 15770 hybrid_clock.cc:648] HybridClock initialized: now 1790211402481201 us; error 53 us; skew 500 ppm
I20260924 00:56:42.481611 15770 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:42 dist-test-slave-mmhs krb5kdc[14592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211402, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.64.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:42.484881 15770 init.cc:377] Logged in from keytab as oryx/127.8.64.131@KRBTEST.COM (short username oryx)
I20260924 00:56:42.486748 15770 webserver.cc:460] Webserver started at http://127.8.64.131:37655/ using document root <none> and password file <none>
I20260924 00:56:42.487720 15770 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:42.487924 15770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:42.493954 15770 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.002s
I20260924 00:56:42.496974 15785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:42.498610 15770 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260924 00:56:42.498797 15770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-2/wal
uuid: "99ac8d2d4cd2408bab694ed9594ef526"
format_stamp: "Formatted at 2026-09-24 00:56:39 on dist-test-slave-mmhs"
server_key: "637afeb5b51098d6c34178ac453d100e"
server_key_iv: "9b429a3a3ce897ac4380142a7510c764"
server_key_version: "encryptionkey@0"
I20260924 00:56:42.499795 15770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:42.525962 15770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:42.530151 15770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:42.530481 15770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:42.531477 15770 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:56:42.532866 15770 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:56:42.532953 15770 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:42.533094 15770 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:56:42.533201 15770 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:42.568511 15770 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.131:33651
I20260924 00:56:42.568606 15897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.131:33651 every 8 connection(s)
I20260924 00:56:42.570380 15770 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
I20260924 00:56:42.576051 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 15770
Sep 24 00:56:42 dist-test-slave-mmhs krb5kdc[14592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211402, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.64.131@KRBTEST.COM for oryx/127.8.64.189@KRBTEST.COM
I20260924 00:56:42.606653 15902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:42.573545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:52515 (local address 127.8.64.189:39347)
0924 00:56:42.574006 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:42.574012 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:42.576656 (+ 2644us) server_negotiation.cc:407] Connection header received
0924 00:56:42.579277 (+ 2621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:42.579287 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:42.579399 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:42.579491 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:42.582708 (+ 3217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:42.584105 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:42.587265 (+ 3160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:42.588126 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:42.588249 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:42.594281 (+ 6032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:42.594324 (+ 43us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:42.594333 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:42.594390 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:42.598565 (+ 4175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:42.599836 (+ 1271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:42.599844 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:42.599862 (+ 18us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:42.600078 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:42.600690 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:42.600698 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:42.600704 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:42.601131 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:42.601269 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:42.603441 (+ 2172us) server_negotiation.cc:299] Negotiation successful
0924 00:56:42.603661 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":621,"threads_started":1}
Sep 24 00:56:42 dist-test-slave-mmhs krb5kdc[14592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211402, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.64.131@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM
Sep 24 00:56:42 dist-test-slave-mmhs krb5kdc[14592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211402, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.64.131@KRBTEST.COM for oryx/127.8.64.188@KRBTEST.COM
I20260924 00:56:42.616504 15905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:42.574280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:39347 (local address 127.8.64.131:52515)
0924 00:56:42.576555 (+ 2275us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:42.576579 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:42.579088 (+ 2509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:42.580046 (+ 958us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:42.580062 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:42.581299 (+ 1237us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:42.582430 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:42.582462 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:42.584344 (+ 1882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:42.584351 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:42.587058 (+ 2707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:42.587079 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:42.588647 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:42.588657 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:42.589034 (+ 377us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:42.590036 (+ 1002us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:42.590075 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:42.593969 (+ 3894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:42.598796 (+ 4827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:42.598809 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:42.598835 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:42.599667 (+ 832us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:42.600245 (+ 578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:42.600253 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:42.600259 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:42.600540 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:42.601360 (+ 820us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:42.601368 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:42.602982 (+ 1614us) client_negotiation.cc:769] Sending connection context
0924 00:56:42.613014 (+ 10032us) client_negotiation.cc:240] Negotiation successful
0924 00:56:42.614573 (+ 1559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2096,"mutex_wait_us":61,"spinlock_wait_cycles":59264,"thread_start_us":1747,"threads_started":1}
I20260924 00:56:42.620429 15900 heartbeater.cc:347] Connected to a master server at 127.8.64.189:39347
I20260924 00:56:42.620791 15900 heartbeater.cc:464] Registering TS with master...
I20260924 00:56:42.621781 15900 heartbeater.cc:511] Master 127.8.64.189:39347 requested a full tablet report, sending...
I20260924 00:56:42.621825 15904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:42.574280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44743 (local address 127.8.64.131:35161)
0924 00:56:42.575419 (+ 1139us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:42.575477 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:42.578650 (+ 3173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:42.601723 (+ 23073us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:42.601728 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:42.602214 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:42.602623 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:42.602640 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:42.604473 (+ 1833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:42.604482 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:42.606285 (+ 1803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:42.606339 (+ 54us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:42.607250 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:42.607258 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:42.607644 (+ 386us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:42.608535 (+ 891us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:42.608566 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:42.612742 (+ 4176us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:42.616996 (+ 4254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:42.617005 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:42.617012 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:42.617626 (+ 614us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:42.618037 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:42.618045 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:42.618051 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:42.618240 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:42.620450 (+ 2210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:42.620459 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:42.620706 (+ 247us) client_negotiation.cc:769] Sending connection context
0924 00:56:42.620896 (+ 190us) client_negotiation.cc:240] Negotiation successful
0924 00:56:42.621202 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":724,"mutex_wait_us":138,"thread_start_us":436,"threads_started":1}
I20260924 00:56:42.624475 15240 ts_manager.cc:194] Registered new tserver with Master: 99ac8d2d4cd2408bab694ed9594ef526 (127.8.64.131:33651)
I20260924 00:56:42.624784 15903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:42.573839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:35161 (local address 127.8.64.190:44743)
0924 00:56:42.601418 (+ 27579us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:42.601424 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:42.601441 (+ 17us) server_negotiation.cc:407] Connection header received
0924 00:56:42.601496 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:42.601500 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:42.601585 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:42.601706 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:42.602777 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:42.604005 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:42.606447 (+ 2442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:42.607048 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:42.607172 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:42.613001 (+ 5829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:42.613008 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:42.613013 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:42.613056 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:42.616718 (+ 3662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:42.617785 (+ 1067us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:42.617793 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:42.617799 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:42.617938 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:42.618389 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:42.618404 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:42.618409 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:42.618830 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:42.618941 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:42.623948 (+ 5007us) server_negotiation.cc:299] Negotiation successful
0924 00:56:42.624285 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27427,"thread_start_us":27989,"threads_started":1}
I20260924 00:56:42.626842 15906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:42.576856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:36561 (local address 127.8.64.131:39975)
0924 00:56:42.605996 (+ 29140us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:42.606016 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:42.606143 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:42.606576 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:42.606580 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:42.607154 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:42.607505 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:42.607524 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:42.608956 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:42.608960 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:42.609981 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:42.609992 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:42.611284 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:42.611289 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:42.611422 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:42.612046 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:42.612063 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:42.614852 (+ 2789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:42.618988 (+ 4136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:42.618991 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:42.618993 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:42.619309 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:42.619776 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:42.619780 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:42.619782 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:42.619904 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:42.625723 (+ 5819us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:42.625733 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:42.625959 (+ 226us) client_negotiation.cc:769] Sending connection context
0924 00:56:42.626145 (+ 186us) client_negotiation.cc:240] Negotiation successful
0924 00:56:42.626443 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29011,"mutex_wait_us":2,"thread_start_us":28865,"threads_started":1}
I20260924 00:56:42.626861 15907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:42.576841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:39975 (local address 127.8.64.188:36561)
0924 00:56:42.605557 (+ 28716us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:42.605563 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:42.606102 (+ 539us) server_negotiation.cc:407] Connection header received
0924 00:56:42.606333 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:42.606339 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:42.606425 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:42.606557 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:42.607662 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:42.608679 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:42.610136 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:42.610767 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:42.610887 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:42.615117 (+ 4230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:42.615125 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:42.615130 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:42.615172 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:42.618670 (+ 3498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:42.619430 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:42.619436 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:42.619440 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:42.619566 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:42.620011 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:42.620017 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:42.620020 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:42.620291 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:42.620435 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:42.626257 (+ 5822us) server_negotiation.cc:299] Negotiation successful
0924 00:56:42.626481 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28597,"thread_start_us":28502,"threads_started":1}
I20260924 00:56:42.627763 15899 heartbeater.cc:347] Connected to a master server at 127.8.64.188:36561
I20260924 00:56:42.627936 15899 heartbeater.cc:464] Registering TS with master...
I20260924 00:56:42.628516 15899 heartbeater.cc:511] Master 127.8.64.188:36561 requested a full tablet report, sending...
I20260924 00:56:42.631155 15429 ts_manager.cc:194] Registered new tserver with Master: 99ac8d2d4cd2408bab694ed9594ef526 (127.8.64.131:33651)
I20260924 00:56:42.632120 15898 heartbeater.cc:347] Connected to a master server at 127.8.64.190:44743
I20260924 00:56:42.632244 15898 heartbeater.cc:464] Registering TS with master...
I20260924 00:56:42.632688 15898 heartbeater.cc:511] Master 127.8.64.190:44743 requested a full tablet report, sending...
I20260924 00:56:42.633694 14637 ts_manager.cc:194] Re-registered known tserver with Master: 99ac8d2d4cd2408bab694ed9594ef526 (127.8.64.131:33651)
I20260924 00:56:42.634907 14637 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.64.131@KRBTEST.COM'} at 127.8.64.131:35161
I20260924 00:56:42.646872 15902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:42.622995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37406 (local address 127.8.64.189:39347)
0924 00:56:42.623227 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:42.623236 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:42.631200 (+ 7964us) server_negotiation.cc:407] Connection header received
0924 00:56:42.631409 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:42.631415 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:42.631534 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:42.631627 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:42.633829 (+ 2202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:42.634957 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:42.636359 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:42.637032 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:42.637172 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:42.640056 (+ 2884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:42.640065 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:42.640070 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:42.640117 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:42.644198 (+ 4081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:42.644829 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:42.644835 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:42.644840 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:42.644968 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:42.645325 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:42.645339 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:42.645343 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:42.645676 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:42.645929 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:42.646281 (+ 352us) server_negotiation.cc:299] Negotiation successful
0924 00:56:42.646536 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260924 00:56:42.654753 15907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:42.632052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43730 (local address 127.8.64.188:36561)
0924 00:56:42.633234 (+ 1182us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:42.633241 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:42.637418 (+ 4177us) server_negotiation.cc:407] Connection header received
0924 00:56:42.637475 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:42.637479 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:42.637549 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:42.637654 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:42.642358 (+ 4704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:42.643169 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:42.645127 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:42.645592 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:42.645727 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:42.649448 (+ 3721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:42.649453 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:42.649456 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:42.649485 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:42.652297 (+ 2812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:42.652916 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:42.652921 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:42.652925 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:42.653048 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:42.653430 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:42.653435 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:42.653438 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:42.653691 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:42.653950 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:42.654272 (+ 322us) server_negotiation.cc:299] Negotiation successful
0924 00:56:42.654503 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1067}
I20260924 00:56:42.657215 15903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:42.615983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39664 (local address 127.8.64.190:44743)
0924 00:56:42.627179 (+ 11196us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:42.627184 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:42.637867 (+ 10683us) server_negotiation.cc:407] Connection header received
0924 00:56:42.637924 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:42.637928 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:42.638008 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:42.638120 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:42.644097 (+ 5977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:42.646073 (+ 1976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:42.648102 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:42.648884 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:42.648984 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:42.652024 (+ 3040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:42.652030 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:42.652036 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:42.652074 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:42.654745 (+ 2671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:42.655481 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:42.655487 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:42.655491 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:42.655613 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:42.655998 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:42.656002 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:42.656004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:42.656260 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:42.656444 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:42.656716 (+ 272us) server_negotiation.cc:299] Negotiation successful
0924 00:56:42.656891 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11074,"thread_start_us":15398,"threads_started":1}
I20260924 00:56:42.660945 14637 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39664:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:56:42.667975 14637 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:56:42.703299 15926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:42.688413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.130:44619 (local address 127.0.0.1:50992)
0924 00:56:42.689036 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:42.689059 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:42.689159 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:42.691529 (+ 2370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:42.691537 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:42.691573 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:56:42.692094 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:42.692109 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:42.695949 (+ 3840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:42.695958 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:42.700139 (+ 4181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:42.700160 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:42.702402 (+ 2242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:42.702409 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:42.702588 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:42.702686 (+ 98us) client_negotiation.cc:769] Sending connection context
0924 00:56:42.702788 (+ 102us) client_negotiation.cc:240] Negotiation successful
0924 00:56:42.702960 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":364,"threads_started":1}
I20260924 00:56:42.704375 15927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:42.688925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50992 (local address 127.8.64.130:44619)
0924 00:56:42.689411 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:42.689421 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:42.689447 (+ 26us) server_negotiation.cc:407] Connection header received
0924 00:56:42.689552 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:42.689560 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:42.689848 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:42.691257 (+ 1409us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:56:42.692321 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:42.695709 (+ 3388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:42.700276 (+ 4567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:42.701817 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:42.701968 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:42.702223 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:42.703719 (+ 1496us) server_negotiation.cc:299] Negotiation successful
0924 00:56:42.704018 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":199,"threads_started":1}
I20260924 00:56:42.709682 15691 tablet_service.cc:1467] Processing CreateTablet for tablet f7e214407ce846e4b42d7e4bd1903d70 (DEFAULT_TABLE table=test-table [id=095b21bf099c4a60945765e28a18cc2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:56:42.711490 15691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7e214407ce846e4b42d7e4bd1903d70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:42.713292 15929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:42.690668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54066 (local address 127.8.64.129:44385)
0924 00:56:42.700148 (+ 9480us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:42.700157 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:42.701334 (+ 1177us) server_negotiation.cc:407] Connection header received
0924 00:56:42.701469 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:42.701475 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:42.701647 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:42.701842 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:56:42.702450 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:42.707879 (+ 5429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:42.709697 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:42.712218 (+ 2521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:42.712383 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:42.712592 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:42.712853 (+ 261us) server_negotiation.cc:299] Negotiation successful
0924 00:56:42.713011 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9314,"thread_start_us":9191,"threads_started":1}
I20260924 00:56:42.713282 15928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:42.689663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:44385 (local address 127.0.0.1:54066)
0924 00:56:42.701200 (+ 11537us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:42.701219 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:42.701344 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:42.701838 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:42.701842 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:42.701867 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:56:42.702227 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:42.702238 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:42.708038 (+ 5800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:42.708043 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:42.709423 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:42.709437 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:42.712443 (+ 3006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:42.712450 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:42.712627 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:42.712661 (+ 34us) client_negotiation.cc:769] Sending connection context
0924 00:56:42.712786 (+ 125us) client_negotiation.cc:240] Negotiation successful
0924 00:56:42.712927 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11413,"thread_start_us":11767,"threads_started":1}
I20260924 00:56:42.716995 15924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:42.687928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.131:33651 (local address 127.0.0.1:60778)
0924 00:56:42.688671 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:42.688697 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:42.688906 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:42.700506 (+ 11600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:42.700509 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:42.700532 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:56:42.700876 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:42.700884 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:42.704597 (+ 3713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:42.704605 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:42.706845 (+ 2240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:42.706862 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:42.716184 (+ 9322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:42.716190 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:42.716360 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:42.716394 (+ 34us) client_negotiation.cc:769] Sending connection context
0924 00:56:42.716496 (+ 102us) client_negotiation.cc:240] Negotiation successful
0924 00:56:42.716655 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":393,"threads_started":1}
I20260924 00:56:42.717063 15925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:42.688266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60778 (local address 127.8.64.131:33651)
0924 00:56:42.699812 (+ 11546us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:42.699824 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:42.699868 (+ 44us) server_negotiation.cc:407] Connection header received
0924 00:56:42.699983 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:42.699993 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:42.700281 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:42.700527 (+ 246us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:56:42.701661 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:42.704036 (+ 2375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:42.714438 (+ 10402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:42.715960 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:42.716107 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:42.716304 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:42.716561 (+ 257us) server_negotiation.cc:299] Negotiation successful
0924 00:56:42.716724 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11344,"thread_start_us":11174,"threads_started":1}
I20260924 00:56:42.716996 15543 tablet_service.cc:1467] Processing CreateTablet for tablet f7e214407ce846e4b42d7e4bd1903d70 (DEFAULT_TABLE table=test-table [id=095b21bf099c4a60945765e28a18cc2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:56:42.718888 15543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7e214407ce846e4b42d7e4bd1903d70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:42.721342 15833 tablet_service.cc:1467] Processing CreateTablet for tablet f7e214407ce846e4b42d7e4bd1903d70 (DEFAULT_TABLE table=test-table [id=095b21bf099c4a60945765e28a18cc2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:56:42.722765 15833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7e214407ce846e4b42d7e4bd1903d70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:42.725206 15930 tablet_bootstrap.cc:492] T f7e214407ce846e4b42d7e4bd1903d70 P 72f23ce899264c60b7b9818e9cc2797c: Bootstrap starting.
I20260924 00:56:42.728434 15930 tablet_bootstrap.cc:654] T f7e214407ce846e4b42d7e4bd1903d70 P 72f23ce899264c60b7b9818e9cc2797c: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:42.729445 15930 log.cc:824] T f7e214407ce846e4b42d7e4bd1903d70 P 72f23ce899264c60b7b9818e9cc2797c: Log is configured to *not* fsync() on all Append() calls
I20260924 00:56:42.732053 15930 tablet_bootstrap.cc:492] T f7e214407ce846e4b42d7e4bd1903d70 P 72f23ce899264c60b7b9818e9cc2797c: No bootstrap required, opened a new log
I20260924 00:56:42.732332 15930 ts_tablet_manager.cc:1397] T f7e214407ce846e4b42d7e4bd1903d70 P 72f23ce899264c60b7b9818e9cc2797c: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260924 00:56:42.735633 15932 tablet_bootstrap.cc:492] T f7e214407ce846e4b42d7e4bd1903d70 P 22c762dcd2e94b6f91721a9571613065: Bootstrap starting.
I20260924 00:56:42.739266 15932 tablet_bootstrap.cc:654] T f7e214407ce846e4b42d7e4bd1903d70 P 22c762dcd2e94b6f91721a9571613065: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:42.739599 15933 tablet_bootstrap.cc:492] T f7e214407ce846e4b42d7e4bd1903d70 P 99ac8d2d4cd2408bab694ed9594ef526: Bootstrap starting.
I20260924 00:56:42.740659 15932 log.cc:824] T f7e214407ce846e4b42d7e4bd1903d70 P 22c762dcd2e94b6f91721a9571613065: Log is configured to *not* fsync() on all Append() calls
I20260924 00:56:42.741909 15930 raft_consensus.cc:348] T f7e214407ce846e4b42d7e4bd1903d70 P 72f23ce899264c60b7b9818e9cc2797c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99ac8d2d4cd2408bab694ed9594ef526" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 33651 } } peers { permanent_uuid: "72f23ce899264c60b7b9818e9cc2797c" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 44619 } } peers { permanent_uuid: "22c762dcd2e94b6f91721a9571613065" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 44385 } }
I20260924 00:56:42.742408 15930 raft_consensus.cc:374] T f7e214407ce846e4b42d7e4bd1903d70 P 72f23ce899264c60b7b9818e9cc2797c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:42.742553 15930 raft_consensus.cc:729] T f7e214407ce846e4b42d7e4bd1903d70 P 72f23ce899264c60b7b9818e9cc2797c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72f23ce899264c60b7b9818e9cc2797c, State: Initialized, Role: FOLLOWER
I20260924 00:56:42.743134 15933 tablet_bootstrap.cc:654] T f7e214407ce846e4b42d7e4bd1903d70 P 99ac8d2d4cd2408bab694ed9594ef526: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:42.743191 15930 consensus_queue.cc:260] T f7e214407ce846e4b42d7e4bd1903d70 P 72f23ce899264c60b7b9818e9cc2797c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99ac8d2d4cd2408bab694ed9594ef526" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 33651 } } peers { permanent_uuid: "72f23ce899264c60b7b9818e9cc2797c" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 44619 } } peers { permanent_uuid: "22c762dcd2e94b6f91721a9571613065" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 44385 } }
I20260924 00:56:42.744288 15757 heartbeater.cc:503] Master 127.8.64.190:44743 was elected leader, sending a full tablet report...
I20260924 00:56:42.745167 15930 ts_tablet_manager.cc:1428] T f7e214407ce846e4b42d7e4bd1903d70 P 72f23ce899264c60b7b9818e9cc2797c: Time spent starting tablet: real 0.013s user 0.004s sys 0.009s
I20260924 00:56:42.745172 15933 log.cc:824] T f7e214407ce846e4b42d7e4bd1903d70 P 99ac8d2d4cd2408bab694ed9594ef526: Log is configured to *not* fsync() on all Append() calls
I20260924 00:56:42.745455 15932 tablet_bootstrap.cc:492] T f7e214407ce846e4b42d7e4bd1903d70 P 22c762dcd2e94b6f91721a9571613065: No bootstrap required, opened a new log
I20260924 00:56:42.746241 15932 ts_tablet_manager.cc:1397] T f7e214407ce846e4b42d7e4bd1903d70 P 22c762dcd2e94b6f91721a9571613065: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260924 00:56:42.754992 15933 tablet_bootstrap.cc:492] T f7e214407ce846e4b42d7e4bd1903d70 P 99ac8d2d4cd2408bab694ed9594ef526: No bootstrap required, opened a new log
I20260924 00:56:42.755355 15933 ts_tablet_manager.cc:1397] T f7e214407ce846e4b42d7e4bd1903d70 P 99ac8d2d4cd2408bab694ed9594ef526: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.009s
I20260924 00:56:42.756844 15932 raft_consensus.cc:348] T f7e214407ce846e4b42d7e4bd1903d70 P 22c762dcd2e94b6f91721a9571613065 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99ac8d2d4cd2408bab694ed9594ef526" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 33651 } } peers { permanent_uuid: "72f23ce899264c60b7b9818e9cc2797c" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 44619 } } peers { permanent_uuid: "22c762dcd2e94b6f91721a9571613065" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 44385 } }
I20260924 00:56:42.757328 15932 raft_consensus.cc:374] T f7e214407ce846e4b42d7e4bd1903d70 P 22c762dcd2e94b6f91721a9571613065 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:42.757433 15932 raft_consensus.cc:729] T f7e214407ce846e4b42d7e4bd1903d70 P 22c762dcd2e94b6f91721a9571613065 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22c762dcd2e94b6f91721a9571613065, State: Initialized, Role: FOLLOWER
I20260924 00:56:42.758108 15932 consensus_queue.cc:260] T f7e214407ce846e4b42d7e4bd1903d70 P 22c762dcd2e94b6f91721a9571613065 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99ac8d2d4cd2408bab694ed9594ef526" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 33651 } } peers { permanent_uuid: "72f23ce899264c60b7b9818e9cc2797c" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 44619 } } peers { permanent_uuid: "22c762dcd2e94b6f91721a9571613065" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 44385 } }
I20260924 00:56:42.760421 15932 ts_tablet_manager.cc:1428] T f7e214407ce846e4b42d7e4bd1903d70 P 22c762dcd2e94b6f91721a9571613065: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
I20260924 00:56:42.765899 15933 raft_consensus.cc:348] T f7e214407ce846e4b42d7e4bd1903d70 P 99ac8d2d4cd2408bab694ed9594ef526 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99ac8d2d4cd2408bab694ed9594ef526" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 33651 } } peers { permanent_uuid: "72f23ce899264c60b7b9818e9cc2797c" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 44619 } } peers { permanent_uuid: "22c762dcd2e94b6f91721a9571613065" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 44385 } }
I20260924 00:56:42.766335 15933 raft_consensus.cc:374] T f7e214407ce846e4b42d7e4bd1903d70 P 99ac8d2d4cd2408bab694ed9594ef526 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:42.766517 15933 raft_consensus.cc:729] T f7e214407ce846e4b42d7e4bd1903d70 P 99ac8d2d4cd2408bab694ed9594ef526 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99ac8d2d4cd2408bab694ed9594ef526, State: Initialized, Role: FOLLOWER
I20260924 00:56:42.767161 15933 consensus_queue.cc:260] T f7e214407ce846e4b42d7e4bd1903d70 P 99ac8d2d4cd2408bab694ed9594ef526 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99ac8d2d4cd2408bab694ed9594ef526" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 33651 } } peers { permanent_uuid: "72f23ce899264c60b7b9818e9cc2797c" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 44619 } } peers { permanent_uuid: "22c762dcd2e94b6f91721a9571613065" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 44385 } }
I20260924 00:56:42.768362 15898 heartbeater.cc:503] Master 127.8.64.190:44743 was elected leader, sending a full tablet report...
I20260924 00:56:42.769028 15933 ts_tablet_manager.cc:1428] T f7e214407ce846e4b42d7e4bd1903d70 P 99ac8d2d4cd2408bab694ed9594ef526: Time spent starting tablet: real 0.014s user 0.010s sys 0.003s
W20260924 00:56:42.824368 15901 tablet.cc:2369] T f7e214407ce846e4b42d7e4bd1903d70 P 99ac8d2d4cd2408bab694ed9594ef526: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 00:56:42.869453 15611 tablet.cc:2369] T f7e214407ce846e4b42d7e4bd1903d70 P 22c762dcd2e94b6f91721a9571613065: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:56:42.957101 15938 raft_consensus.cc:482] T f7e214407ce846e4b42d7e4bd1903d70 P 99ac8d2d4cd2408bab694ed9594ef526 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:56:42.957535 15938 raft_consensus.cc:504] T f7e214407ce846e4b42d7e4bd1903d70 P 99ac8d2d4cd2408bab694ed9594ef526 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99ac8d2d4cd2408bab694ed9594ef526" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 33651 } } peers { permanent_uuid: "72f23ce899264c60b7b9818e9cc2797c" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 44619 } } peers { permanent_uuid: "22c762dcd2e94b6f91721a9571613065" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 44385 } }
I20260924 00:56:42.959342 15938 leader_election.cc:288] T f7e214407ce846e4b42d7e4bd1903d70 P 99ac8d2d4cd2408bab694ed9594ef526 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72f23ce899264c60b7b9818e9cc2797c (127.8.64.130:44619), 22c762dcd2e94b6f91721a9571613065 (127.8.64.129:44385)
I20260924 00:56:42.965766 15906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:42.959728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.130:44619 (local address 127.8.64.131:52715)
0924 00:56:42.959974 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:42.959995 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:42.960197 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:42.960743 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:42.960748 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:42.960772 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:56:42.960998 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:42.961009 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:42.962048 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:42.962053 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:42.963499 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:42.963511 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:42.964976 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:42.964981 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:42.965214 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:42.965248 (+ 34us) client_negotiation.cc:769] Sending connection context
0924 00:56:42.965345 (+ 97us) client_negotiation.cc:240] Negotiation successful
0924 00:56:42.965473 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260924 00:56:42.965816 15927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:42.959798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:52715 (local address 127.8.64.130:44619)
0924 00:56:42.959993 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:42.959999 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:42.960317 (+ 318us) server_negotiation.cc:407] Connection header received
0924 00:56:42.960401 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:42.960406 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:42.960474 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:42.960565 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:56:42.961165 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:42.961909 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:42.963675 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:42.964647 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:42.964801 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:42.964878 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:42.965388 (+ 510us) server_negotiation.cc:299] Negotiation successful
0924 00:56:42.965534 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260924 00:56:42.966907 15905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:42.960203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:44385 (local address 127.8.64.131:58647)
0924 00:56:42.960410 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:42.960426 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:42.960525 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:42.960870 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:42.960874 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:42.960913 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:56:42.961117 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:42.961136 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:42.962577 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:42.962582 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:42.964542 (+ 1960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:42.964556 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:42.966119 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:42.966125 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:42.966357 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:42.966391 (+ 34us) client_negotiation.cc:769] Sending connection context
0924 00:56:42.966526 (+ 135us) client_negotiation.cc:240] Negotiation successful
0924 00:56:42.966660 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101}
I20260924 00:56:42.967001 15929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:42.960314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:58647 (local address 127.8.64.129:44385)
0924 00:56:42.960503 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:42.960509 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:42.960527 (+ 18us) server_negotiation.cc:407] Connection header received
0924 00:56:42.960689 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:42.960699 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:42.960754 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:42.960853 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:56:42.961302 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:42.962395 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:42.964713 (+ 2318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:42.965963 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:42.966095 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:42.966173 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:42.966574 (+ 401us) server_negotiation.cc:299] Negotiation successful
0924 00:56:42.966703 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260924 00:56:42.967288 15711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7e214407ce846e4b42d7e4bd1903d70" candidate_uuid: "99ac8d2d4cd2408bab694ed9594ef526" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72f23ce899264c60b7b9818e9cc2797c" is_pre_election: true
I20260924 00:56:42.967952 15563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7e214407ce846e4b42d7e4bd1903d70" candidate_uuid: "99ac8d2d4cd2408bab694ed9594ef526" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22c762dcd2e94b6f91721a9571613065" is_pre_election: true
W20260924 00:56:42.968358 15760 tablet.cc:2369] T f7e214407ce846e4b42d7e4bd1903d70 P 72f23ce899264c60b7b9818e9cc2797c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:56:42.968454 15711 raft_consensus.cc:2454] T f7e214407ce846e4b42d7e4bd1903d70 P 72f23ce899264c60b7b9818e9cc2797c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99ac8d2d4cd2408bab694ed9594ef526 in term 0.
I20260924 00:56:42.968698 15563 raft_consensus.cc:2454] T f7e214407ce846e4b42d7e4bd1903d70 P 22c762dcd2e94b6f91721a9571613065 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99ac8d2d4cd2408bab694ed9594ef526 in term 0.
I20260924 00:56:42.969441 15788 leader_election.cc:302] T f7e214407ce846e4b42d7e4bd1903d70 P 99ac8d2d4cd2408bab694ed9594ef526 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22c762dcd2e94b6f91721a9571613065, 99ac8d2d4cd2408bab694ed9594ef526; no voters:
I20260924 00:56:42.969856 15938 raft_consensus.cc:2781] T f7e214407ce846e4b42d7e4bd1903d70 P 99ac8d2d4cd2408bab694ed9594ef526 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:56:42.970039 15938 raft_consensus.cc:482] T f7e214407ce846e4b42d7e4bd1903d70 P 99ac8d2d4cd2408bab694ed9594ef526 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:56:42.970196 15938 raft_consensus.cc:3037] T f7e214407ce846e4b42d7e4bd1903d70 P 99ac8d2d4cd2408bab694ed9594ef526 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:42.972574 15938 raft_consensus.cc:504] T f7e214407ce846e4b42d7e4bd1903d70 P 99ac8d2d4cd2408bab694ed9594ef526 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99ac8d2d4cd2408bab694ed9594ef526" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 33651 } } peers { permanent_uuid: "72f23ce899264c60b7b9818e9cc2797c" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 44619 } } peers { permanent_uuid: "22c762dcd2e94b6f91721a9571613065" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 44385 } }
I20260924 00:56:42.973150 15938 leader_election.cc:288] T f7e214407ce846e4b42d7e4bd1903d70 P 99ac8d2d4cd2408bab694ed9594ef526 [CANDIDATE]: Term 1 election: Requested vote from peers 72f23ce899264c60b7b9818e9cc2797c (127.8.64.130:44619), 22c762dcd2e94b6f91721a9571613065 (127.8.64.129:44385)
I20260924 00:56:42.973668 15711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7e214407ce846e4b42d7e4bd1903d70" candidate_uuid: "99ac8d2d4cd2408bab694ed9594ef526" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72f23ce899264c60b7b9818e9cc2797c"
I20260924 00:56:42.973655 15563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7e214407ce846e4b42d7e4bd1903d70" candidate_uuid: "99ac8d2d4cd2408bab694ed9594ef526" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22c762dcd2e94b6f91721a9571613065"
I20260924 00:56:42.973837 15563 raft_consensus.cc:3037] T f7e214407ce846e4b42d7e4bd1903d70 P 22c762dcd2e94b6f91721a9571613065 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:42.973881 15711 raft_consensus.cc:3037] T f7e214407ce846e4b42d7e4bd1903d70 P 72f23ce899264c60b7b9818e9cc2797c [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:42.976737 15711 raft_consensus.cc:2454] T f7e214407ce846e4b42d7e4bd1903d70 P 72f23ce899264c60b7b9818e9cc2797c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99ac8d2d4cd2408bab694ed9594ef526 in term 1.
I20260924 00:56:42.977051 15563 raft_consensus.cc:2454] T f7e214407ce846e4b42d7e4bd1903d70 P 22c762dcd2e94b6f91721a9571613065 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99ac8d2d4cd2408bab694ed9594ef526 in term 1.
I20260924 00:56:42.977229 15788 leader_election.cc:302] T f7e214407ce846e4b42d7e4bd1903d70 P 99ac8d2d4cd2408bab694ed9594ef526 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72f23ce899264c60b7b9818e9cc2797c, 99ac8d2d4cd2408bab694ed9594ef526; no voters:
I20260924 00:56:42.977578 15938 raft_consensus.cc:2781] T f7e214407ce846e4b42d7e4bd1903d70 P 99ac8d2d4cd2408bab694ed9594ef526 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:56:42.978087 15938 raft_consensus.cc:686] T f7e214407ce846e4b42d7e4bd1903d70 P 99ac8d2d4cd2408bab694ed9594ef526 [term 1 LEADER]: Becoming Leader. State: Replica: 99ac8d2d4cd2408bab694ed9594ef526, State: Running, Role: LEADER
I20260924 00:56:42.978927 15938 consensus_queue.cc:237] T f7e214407ce846e4b42d7e4bd1903d70 P 99ac8d2d4cd2408bab694ed9594ef526 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99ac8d2d4cd2408bab694ed9594ef526" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 33651 } } peers { permanent_uuid: "72f23ce899264c60b7b9818e9cc2797c" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 44619 } } peers { permanent_uuid: "22c762dcd2e94b6f91721a9571613065" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 44385 } }
I20260924 00:56:42.984079 14637 catalog_manager.cc:5495] T f7e214407ce846e4b42d7e4bd1903d70 P 99ac8d2d4cd2408bab694ed9594ef526 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99ac8d2d4cd2408bab694ed9594ef526 (127.8.64.131). New cstate: current_term: 1 leader_uuid: "99ac8d2d4cd2408bab694ed9594ef526" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99ac8d2d4cd2408bab694ed9594ef526" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 33651 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72f23ce899264c60b7b9818e9cc2797c" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 44619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22c762dcd2e94b6f91721a9571613065" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 44385 } health_report { overall_health: UNKNOWN } } }
I20260924 00:56:43.064831 15925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:43.059033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60780 (local address 127.8.64.131:33651)
0924 00:56:43.059234 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:43.059240 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:43.059260 (+ 20us) server_negotiation.cc:407] Connection header received
0924 00:56:43.059466 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:43.059469 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:43.059544 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:43.059697 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:56:43.060201 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:43.060973 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:43.062476 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:43.063057 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:43.063214 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:43.063730 (+ 516us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:56:43.064230 (+ 500us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:56:43.064402 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:43.064452 (+ 50us) server_negotiation.cc:299] Negotiation successful
0924 00:56:43.064532 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260924 00:56:43.074452 15563 raft_consensus.cc:1261] T f7e214407ce846e4b42d7e4bd1903d70 P 22c762dcd2e94b6f91721a9571613065 [term 1 FOLLOWER]: Refusing update from remote peer 99ac8d2d4cd2408bab694ed9594ef526: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:56:43.074555 15711 raft_consensus.cc:1261] T f7e214407ce846e4b42d7e4bd1903d70 P 72f23ce899264c60b7b9818e9cc2797c [term 1 FOLLOWER]: Refusing update from remote peer 99ac8d2d4cd2408bab694ed9594ef526: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:56:43.075690 15939 consensus_queue.cc:1035] T f7e214407ce846e4b42d7e4bd1903d70 P 99ac8d2d4cd2408bab694ed9594ef526 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22c762dcd2e94b6f91721a9571613065" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 44385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:56:43.076160 15938 consensus_queue.cc:1035] T f7e214407ce846e4b42d7e4bd1903d70 P 99ac8d2d4cd2408bab694ed9594ef526 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72f23ce899264c60b7b9818e9cc2797c" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 44619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:56:43.089594 15942 mvcc.cc:205] Tried to move back new op lower bound from 7332705906981294080 to 7332705906605715456. Current Snapshot: MvccSnapshot[applied={T|T < 7332705906981294080}]
I20260924 00:56:43.095350 15943 mvcc.cc:205] Tried to move back new op lower bound from 7332705906981294080 to 7332705906605715456. Current Snapshot: MvccSnapshot[applied={T|T < 7332705906981294080}]
I20260924 00:56:43.101583 15944 mvcc.cc:205] Tried to move back new op lower bound from 7332705906981294080 to 7332705906605715456. Current Snapshot: MvccSnapshot[applied={T|T < 7332705906981294080}]
I20260924 00:56:43.120810 14637 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:56:43.121124 14637 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:56:43.127862 14637 catalog_manager.cc:5782] T 00000000000000000000000000000000 P db882bdd18eb4243928039daa8c49303: Sending DeleteTablet for 3 replicas of tablet f7e214407ce846e4b42d7e4bd1903d70
I20260924 00:56:43.129026 15833 tablet_service.cc:1514] Processing DeleteTablet for tablet f7e214407ce846e4b42d7e4bd1903d70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:56:43 UTC) from {username='oryx', principal='oryx/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:60778
I20260924 00:56:43.130406 15953 tablet_replica.cc:330] T f7e214407ce846e4b42d7e4bd1903d70 P 99ac8d2d4cd2408bab694ed9594ef526: stopping tablet replica
I20260924 00:56:43.130612 15691 tablet_service.cc:1514] Processing DeleteTablet for tablet f7e214407ce846e4b42d7e4bd1903d70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:56:43 UTC) from {username='oryx', principal='oryx/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:50992
I20260924 00:56:43.131179 15953 raft_consensus.cc:2229] T f7e214407ce846e4b42d7e4bd1903d70 P 99ac8d2d4cd2408bab694ed9594ef526 [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:56:43.131906 15953 raft_consensus.cc:2258] T f7e214407ce846e4b42d7e4bd1903d70 P 99ac8d2d4cd2408bab694ed9594ef526 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:56:43.132287 15543 tablet_service.cc:1514] Processing DeleteTablet for tablet f7e214407ce846e4b42d7e4bd1903d70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:56:43 UTC) from {username='oryx', principal='oryx/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:54066
I20260924 00:56:43.133366 15954 tablet_replica.cc:330] T f7e214407ce846e4b42d7e4bd1903d70 P 72f23ce899264c60b7b9818e9cc2797c: stopping tablet replica
I20260924 00:56:43.133852 15954 raft_consensus.cc:2229] T f7e214407ce846e4b42d7e4bd1903d70 P 72f23ce899264c60b7b9818e9cc2797c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:56:43.134508 15954 raft_consensus.cc:2258] T f7e214407ce846e4b42d7e4bd1903d70 P 72f23ce899264c60b7b9818e9cc2797c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:56:43.134637 15953 ts_tablet_manager.cc:1905] T f7e214407ce846e4b42d7e4bd1903d70 P 99ac8d2d4cd2408bab694ed9594ef526: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:56:43.136641 15954 ts_tablet_manager.cc:1905] T f7e214407ce846e4b42d7e4bd1903d70 P 72f23ce899264c60b7b9818e9cc2797c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:56:43.141862 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 14605
I20260924 00:56:43.143213 14667 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:56:43.144773 15954 ts_tablet_manager.cc:1918] T f7e214407ce846e4b42d7e4bd1903d70 P 72f23ce899264c60b7b9818e9cc2797c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:56:43.145059 15954 log.cc:1196] T f7e214407ce846e4b42d7e4bd1903d70 P 72f23ce899264c60b7b9818e9cc2797c: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-1/wal/wals/f7e214407ce846e4b42d7e4bd1903d70
I20260924 00:56:43.145880 15954 ts_tablet_manager.cc:1939] T f7e214407ce846e4b42d7e4bd1903d70 P 72f23ce899264c60b7b9818e9cc2797c: Deleting consensus metadata
I20260924 00:56:43.150368 15953 ts_tablet_manager.cc:1918] T f7e214407ce846e4b42d7e4bd1903d70 P 99ac8d2d4cd2408bab694ed9594ef526: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:56:43.151473 15955 tablet_replica.cc:330] T f7e214407ce846e4b42d7e4bd1903d70 P 22c762dcd2e94b6f91721a9571613065: stopping tablet replica
I20260924 00:56:43.152015 15953 log.cc:1196] T f7e214407ce846e4b42d7e4bd1903d70 P 99ac8d2d4cd2408bab694ed9594ef526: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-2/wal/wals/f7e214407ce846e4b42d7e4bd1903d70
I20260924 00:56:43.152257 15955 raft_consensus.cc:2229] T f7e214407ce846e4b42d7e4bd1903d70 P 22c762dcd2e94b6f91721a9571613065 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:56:43.152668 15953 ts_tablet_manager.cc:1939] T f7e214407ce846e4b42d7e4bd1903d70 P 99ac8d2d4cd2408bab694ed9594ef526: Deleting consensus metadata
I20260924 00:56:43.153033 15955 raft_consensus.cc:2258] T f7e214407ce846e4b42d7e4bd1903d70 P 22c762dcd2e94b6f91721a9571613065 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:56:43.155797 15955 ts_tablet_manager.cc:1905] T f7e214407ce846e4b42d7e4bd1903d70 P 22c762dcd2e94b6f91721a9571613065: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:56:43.166000 15955 ts_tablet_manager.cc:1918] T f7e214407ce846e4b42d7e4bd1903d70 P 22c762dcd2e94b6f91721a9571613065: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:56:43.166448 15955 log.cc:1196] T f7e214407ce846e4b42d7e4bd1903d70 P 22c762dcd2e94b6f91721a9571613065: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-0/wal/wals/f7e214407ce846e4b42d7e4bd1903d70
I20260924 00:56:43.167335 15955 ts_tablet_manager.cc:1939] T f7e214407ce846e4b42d7e4bd1903d70 P 22c762dcd2e94b6f91721a9571613065: Deleting consensus metadata
I20260924 00:56:43.170675 15902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:43.138265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37410 (local address 127.8.64.189:39347)
0924 00:56:43.138462 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:43.138468 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:43.141056 (+ 2588us) server_negotiation.cc:407] Connection header received
0924 00:56:43.150724 (+ 9668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:43.150731 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:43.150820 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:43.150891 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:43.153567 (+ 2676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:43.154958 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:43.157082 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:43.157821 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:43.157919 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:43.162152 (+ 4233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:43.162159 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:43.162164 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:43.162207 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:43.166448 (+ 4241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:43.167397 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:43.167403 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:43.167409 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:43.167592 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:43.168319 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:43.168325 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:43.168330 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:43.168709 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:43.169017 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:43.169829 (+ 812us) server_negotiation.cc:299] Negotiation successful
0924 00:56:43.170148 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260924 00:56:43.195623 15907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:43.141248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43740 (local address 127.8.64.188:36561)
0924 00:56:43.141398 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:43.141403 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:43.171940 (+ 30537us) server_negotiation.cc:407] Connection header received
0924 00:56:43.172197 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:43.172204 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:43.172310 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:43.172384 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:43.174737 (+ 2353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:43.176016 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:43.178152 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:43.178875 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:43.179003 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:43.183226 (+ 4223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:43.183234 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:43.183241 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:43.183299 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:43.187812 (+ 4513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:43.192376 (+ 4564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:43.192382 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:43.192386 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:43.192574 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:43.193106 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:43.193111 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:43.193115 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:43.193538 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:43.193782 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:43.194957 (+ 1175us) server_negotiation.cc:299] Negotiation successful
0924 00:56:43.195174 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 00:56:43.303079 14622 catalog_manager.cc:4841] TS 99ac8d2d4cd2408bab694ed9594ef526 (127.8.64.131:33651): tablet f7e214407ce846e4b42d7e4bd1903d70 (table test-table [id=095b21bf099c4a60945765e28a18cc2a]) successfully deleted
I20260924 00:56:43.304312 14623 catalog_manager.cc:4841] TS 72f23ce899264c60b7b9818e9cc2797c (127.8.64.130:44619): tablet f7e214407ce846e4b42d7e4bd1903d70 (table test-table [id=095b21bf099c4a60945765e28a18cc2a]) successfully deleted
W20260924 00:56:43.304561 14623 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.8.64.130:44619, user_credentials={real_user=slave}} blocked reactor thread for 156555us
I20260924 00:56:43.305315 14623 catalog_manager.cc:4841] TS 22c762dcd2e94b6f91721a9571613065 (127.8.64.129:44385): tablet f7e214407ce846e4b42d7e4bd1903d70 (table test-table [id=095b21bf099c4a60945765e28a18cc2a]) successfully deleted
I20260924 00:56:43.305454 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 14605
W20260924 00:56:43.304241 14622 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.8.64.131:33651, user_credentials={real_user=slave}} blocked reactor thread for 149093us
I20260924 00:56:43.356724 15902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:43.339539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37426 (local address 127.8.64.189:39347)
0924 00:56:43.339816 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:43.339825 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:43.340807 (+ 982us) server_negotiation.cc:407] Connection header received
0924 00:56:43.341036 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:43.341045 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:43.341135 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:43.341213 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:43.343741 (+ 2528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:43.344937 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:43.346196 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:43.346888 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:43.347016 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:43.350519 (+ 3503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:43.350526 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:43.350532 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:43.350574 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:43.353661 (+ 3087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:43.354406 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:43.354413 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:43.354417 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:43.354591 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:43.355059 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:43.355065 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:43.355071 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:43.355495 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:43.355739 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:43.356214 (+ 475us) server_negotiation.cc:299] Negotiation successful
0924 00:56:43.356425 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260924 00:56:43.357745 15907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:43.340908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43754 (local address 127.8.64.188:36561)
0924 00:56:43.341104 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:43.341111 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:43.341351 (+ 240us) server_negotiation.cc:407] Connection header received
0924 00:56:43.341496 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:43.341502 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:43.341577 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:43.341678 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:43.345128 (+ 3450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:43.346232 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:43.347916 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:43.348319 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:43.348448 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:43.351035 (+ 2587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:43.351040 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:43.351045 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:43.351084 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:43.354278 (+ 3194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:43.355127 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:43.355134 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:43.355138 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:43.355270 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:43.355667 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:43.355672 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:43.355676 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:43.356028 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:43.356245 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:43.356921 (+ 676us) server_negotiation.cc:299] Negotiation successful
0924 00:56:43.357136 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260924 00:56:44.104422 15964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:44.103638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44743 (local address 127.8.64.130:50885)
0924 00:56:44.104108 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:44.104215 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:44743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":225,"threads_started":1}
W20260924 00:56:44.104847 15757 heartbeater.cc:644] Failed to heartbeat to 127.8.64.190:44743 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.64.190:44743: connect: Connection refused (error 111)
I20260924 00:56:44.110297 15965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:44.109587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44743 (local address 127.8.64.129:49715)
0924 00:56:44.109994 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:44.110093 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:44743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":223,"threads_started":1}
W20260924 00:56:44.110687 15608 heartbeater.cc:644] Failed to heartbeat to 127.8.64.190:44743 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.64.190:44743: connect: Connection refused (error 111)
I20260924 00:56:44.114013 15966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:44.113234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44743 (local address 127.8.64.131:50287)
0924 00:56:44.113673 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:44.113803 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:44743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":230,"threads_started":1}
W20260924 00:56:44.114379 15898 heartbeater.cc:644] Failed to heartbeat to 127.8.64.190:44743 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.64.190:44743: connect: Connection refused (error 111)
I20260924 00:56:44.629714 15967 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader db882bdd18eb4243928039daa8c49303)
I20260924 00:56:44.630105 15967 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } peers { permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 39347 } attrs { promote: false } } peers { permanent_uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36561 } attrs { promote: false } }
I20260924 00:56:44.630373 15968 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader db882bdd18eb4243928039daa8c49303)
I20260924 00:56:44.630607 15968 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } peers { permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 39347 } attrs { promote: false } } peers { permanent_uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36561 } attrs { promote: false } }
I20260924 00:56:44.631771 15967 leader_election.cc:288] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers db882bdd18eb4243928039daa8c49303 (127.8.64.190:44743), 52ef84aecd3d45dd9caa57e3f2cc0d86 (127.8.64.188:36561)
I20260924 00:56:44.631901 15968 leader_election.cc:288] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers db882bdd18eb4243928039daa8c49303 (127.8.64.190:44743), 1d362bb4f4dd41adaee4592166347b34 (127.8.64.189:39347)
I20260924 00:56:44.633042 15969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:44.631839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44743 (local address 127.0.0.1:52932)
0924 00:56:44.632424 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:44.632650 (+ 226us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:44743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":313,"threads_started":1}
W20260924 00:56:44.633507 15225 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.64.190:44743: connect: Connection refused (error 111)
I20260924 00:56:44.642597 15974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:44.631964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44743 (local address 127.0.0.1:52940)
0924 00:56:44.641927 (+ 9963us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:44.642210 (+ 283us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:44743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":9798,"threads_started":1}
I20260924 00:56:44.646577 15971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:44.645115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44743 (local address 127.0.0.1:52946)
0924 00:56:44.646038 (+ 923us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:44.646254 (+ 216us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:44743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":765}
W20260924 00:56:44.646924 15225 leader_election.cc:334] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer db882bdd18eb4243928039daa8c49303 (127.8.64.190:44743): Network error: Client connection negotiation failed: client connection to 127.8.64.190:44743: connect: Connection refused (error 111)
W20260924 00:56:44.648020 15413 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.64.190:44743: connect: Connection refused (error 111)
I20260924 00:56:44.649923 15970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:44.649378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44743 (local address 127.0.0.1:52952)
0924 00:56:44.649551 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:44.649673 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:44743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260924 00:56:44.650177 15413 leader_election.cc:334] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer db882bdd18eb4243928039daa8c49303 (127.8.64.190:44743): Network error: Client connection negotiation failed: client connection to 127.8.64.190:44743: connect: Connection refused (error 111)
I20260924 00:56:44.651945 15969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:44.632307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:36561 (local address 127.0.0.1:48996)
0924 00:56:44.639236 (+ 6929us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:44.639268 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:44.639471 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:44.645206 (+ 5735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:44.645214 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:44.645256 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:56:44.645651 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:44.645667 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:44.647731 (+ 2064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:44.647736 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:44.649639 (+ 1903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:44.649650 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:44.651408 (+ 1758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:44.651412 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:44.651534 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:44.651559 (+ 25us) client_negotiation.cc:769] Sending connection context
0924 00:56:44.651637 (+ 78us) client_negotiation.cc:240] Negotiation successful
0924 00:56:44.651732 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6502,"thread_start_us":13122,"threads_started":1}
I20260924 00:56:44.652196 15972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:44.632826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48996 (local address 127.8.64.188:36561)
0924 00:56:44.644643 (+ 11817us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:44.644651 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:44.644673 (+ 22us) server_negotiation.cc:407] Connection header received
0924 00:56:44.644748 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:44.644756 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:44.644846 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:44.645556 (+ 710us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:56:44.645803 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:44.647537 (+ 1734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:44.649790 (+ 2253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:44.651247 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:44.651377 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:44.651476 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:44.651679 (+ 203us) server_negotiation.cc:299] Negotiation successful
0924 00:56:44.651806 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":11666,"thread_start_us":12142,"threads_started":1}
I20260924 00:56:44.652791 15439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1d362bb4f4dd41adaee4592166347b34" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86" is_pre_election: true
I20260924 00:56:44.653146 15439 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d362bb4f4dd41adaee4592166347b34 in term 1.
I20260924 00:56:44.653710 15227 leader_election.cc:302] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1d362bb4f4dd41adaee4592166347b34, 52ef84aecd3d45dd9caa57e3f2cc0d86; no voters: db882bdd18eb4243928039daa8c49303
I20260924 00:56:44.653992 15974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:44.637434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:39347 (local address 127.0.0.1:59986)
0924 00:56:44.644168 (+ 6734us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:44.644192 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:44.644357 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:44.648006 (+ 3649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:44.648011 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:44.648060 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:56:44.648386 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:44.648402 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:44.649880 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:44.649885 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:44.651876 (+ 1991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:44.651888 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:44.653182 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:44.653188 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:44.653376 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:44.653414 (+ 38us) client_negotiation.cc:769] Sending connection context
0924 00:56:44.653527 (+ 113us) client_negotiation.cc:240] Negotiation successful
0924 00:56:44.653653 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6603,"mutex_wait_us":25,"thread_start_us":4189,"threads_started":1}
I20260924 00:56:44.654093 15975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:44.645708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59986 (local address 127.8.64.189:39347)
0924 00:56:44.647696 (+ 1988us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:44.647704 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:44.647729 (+ 25us) server_negotiation.cc:407] Connection header received
0924 00:56:44.647792 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:44.647802 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:44.647857 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:44.647988 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:56:44.648562 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:44.649771 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:44.652044 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:44.652963 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:44.653100 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:44.653171 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:44.653564 (+ 393us) server_negotiation.cc:299] Negotiation successful
0924 00:56:44.653675 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":1842,"thread_start_us":1691,"threads_started":1}
I20260924 00:56:44.654170 15967 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 00:56:44.654277 15967 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [term 1 FOLLOWER]: Starting leader election (detected failure of leader db882bdd18eb4243928039daa8c49303)
I20260924 00:56:44.654434 15967 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [term 1 FOLLOWER]: Advancing to term 2
I20260924 00:56:44.654883 15250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1d362bb4f4dd41adaee4592166347b34" is_pre_election: true
I20260924 00:56:44.656718 15967 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } peers { permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 39347 } attrs { promote: false } } peers { permanent_uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36561 } attrs { promote: false } }
I20260924 00:56:44.657112 15250 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ef84aecd3d45dd9caa57e3f2cc0d86 in current term 2: Already voted for candidate 1d362bb4f4dd41adaee4592166347b34 in this term.
I20260924 00:56:44.657456 15967 leader_election.cc:288] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [CANDIDATE]: Term 2 election: Requested vote from peers db882bdd18eb4243928039daa8c49303 (127.8.64.190:44743), 52ef84aecd3d45dd9caa57e3f2cc0d86 (127.8.64.188:36561)
I20260924 00:56:44.657898 15439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1d362bb4f4dd41adaee4592166347b34" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86"
I20260924 00:56:44.657984 15969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:44.657532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44743 (local address 127.0.0.1:52958)
0924 00:56:44.657722 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:44.657809 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:44743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260924 00:56:44.658135 15439 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86 [term 1 FOLLOWER]: Advancing to term 2
I20260924 00:56:44.658201 15415 leader_election.cc:302] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ef84aecd3d45dd9caa57e3f2cc0d86; no voters: 1d362bb4f4dd41adaee4592166347b34, db882bdd18eb4243928039daa8c49303
I20260924 00:56:44.659010 15969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:44.658664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44743 (local address 127.0.0.1:52964)
0924 00:56:44.658798 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:44.658889 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:44743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260924 00:56:44.659307 15225 leader_election.cc:334] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer db882bdd18eb4243928039daa8c49303 (127.8.64.190:44743): Network error: Client connection negotiation failed: client connection to 127.8.64.190:44743: connect: Connection refused (error 111)
I20260924 00:56:44.660583 15439 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d362bb4f4dd41adaee4592166347b34 in term 2.
I20260924 00:56:44.660748 15968 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 00:56:44.661077 15227 leader_election.cc:302] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1d362bb4f4dd41adaee4592166347b34, 52ef84aecd3d45dd9caa57e3f2cc0d86; no voters: db882bdd18eb4243928039daa8c49303
I20260924 00:56:44.661373 15967 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 00:56:44.661821 15967 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [term 2 LEADER]: Becoming Leader. State: Replica: 1d362bb4f4dd41adaee4592166347b34, State: Running, Role: LEADER
I20260924 00:56:44.662180 15967 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } peers { permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 39347 } attrs { promote: false } } peers { permanent_uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36561 } attrs { promote: false } }
I20260924 00:56:44.663986 15977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d362bb4f4dd41adaee4592166347b34. Latest consensus state: current_term: 2 leader_uuid: "1d362bb4f4dd41adaee4592166347b34" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } peers { permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 39347 } attrs { promote: false } } peers { permanent_uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36561 } attrs { promote: false } } }
I20260924 00:56:44.664397 15977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [sys.catalog]: This master's current role is: LEADER
I20260924 00:56:44.665011 15979 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:56:44.668412 15979 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:56:44.669209 15979 catalog_manager.cc:1259] Loaded cluster ID: d3b5b249561e46889e63beb4a75853a8
I20260924 00:56:44.669275 15979 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:56:44.670354 15979 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:56:44.671109 15979 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34: Loaded TSK: 0
I20260924 00:56:44.671890 15979 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 00:56:44.692577 15240 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37426:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:56:44.699622 15240 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:56:44.703956 15969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:44.703504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44743 (local address 127.0.0.1:52988)
0924 00:56:44.703728 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:44.703811 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:44743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260924 00:56:44.704049 15439 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86 [term 2 FOLLOWER]: Refusing update from remote peer 1d362bb4f4dd41adaee4592166347b34: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260924 00:56:44.704851 15977 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36561 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260924 00:56:44.705116 15969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:44.704736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44743 (local address 127.0.0.1:53004)
0924 00:56:44.704893 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:44.704960 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:44743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260924 00:56:44.705265 15225 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 -> Peer db882bdd18eb4243928039daa8c49303 (127.8.64.190:44743): Couldn't send request to peer db882bdd18eb4243928039daa8c49303. Status: Network error: Client connection negotiation failed: client connection to 127.8.64.190:44743: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 00:56:44.707873 15968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d362bb4f4dd41adaee4592166347b34. Latest consensus state: current_term: 2 leader_uuid: "1d362bb4f4dd41adaee4592166347b34" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } peers { permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 39347 } attrs { promote: false } } peers { permanent_uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36561 } attrs { promote: false } } }
I20260924 00:56:44.708124 15968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:56:44.709523 15977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "1d362bb4f4dd41adaee4592166347b34" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } peers { permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 39347 } attrs { promote: false } } peers { permanent_uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36561 } attrs { promote: false } } }
I20260924 00:56:44.709824 15977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34 [sys.catalog]: This master's current role is: LEADER
I20260924 00:56:44.709939 15968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "1d362bb4f4dd41adaee4592166347b34" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "db882bdd18eb4243928039daa8c49303" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44743 } } peers { permanent_uuid: "1d362bb4f4dd41adaee4592166347b34" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 39347 } attrs { promote: false } } peers { permanent_uuid: "52ef84aecd3d45dd9caa57e3f2cc0d86" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36561 } attrs { promote: false } } }
I20260924 00:56:44.710125 15981 mvcc.cc:205] Tried to move back new op lower bound from 7332705913660104704 to 7332705913499467776. Current Snapshot: MvccSnapshot[applied={T|T < 7332705913660104704}]
I20260924 00:56:44.710201 15968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52ef84aecd3d45dd9caa57e3f2cc0d86 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:56:44.724814 15969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:44.718589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.131:33651 (local address 127.0.0.1:35896)
0924 00:56:44.719467 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:44.719484 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:44.719571 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:44.719930 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:44.719936 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:44.719968 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:56:44.720194 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:44.720208 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:44.721378 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:44.721384 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:44.723017 (+ 1633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:44.723029 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:44.724081 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:44.724087 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:44.724250 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:44.724273 (+ 23us) client_negotiation.cc:769] Sending connection context
0924 00:56:44.724377 (+ 104us) client_negotiation.cc:240] Negotiation successful
0924 00:56:44.724508 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":697}
I20260924 00:56:44.724820 15987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:44.719128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35896 (local address 127.8.64.131:33651)
0924 00:56:44.719557 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:44.719563 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:44.719581 (+ 18us) server_negotiation.cc:407] Connection header received
0924 00:56:44.719679 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:44.719684 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:44.719756 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:44.720417 (+ 661us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:56:44.720576 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:44.721231 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:44.723165 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:44.723921 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:44.724056 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:44.724156 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:44.724412 (+ 256us) server_negotiation.cc:299] Negotiation successful
0924 00:56:44.724542 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":603,"threads_started":1}
I20260924 00:56:44.726112 15833 tablet_service.cc:1467] Processing CreateTablet for tablet 98480b849b974b3d9aea2f30c0c6408b (DEFAULT_TABLE table=test-table [id=708efb5782c048a78ea0b0201ef30059]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:56:44.726719 15833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98480b849b974b3d9aea2f30c0c6408b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:44.733727 15990 tablet_bootstrap.cc:492] T 98480b849b974b3d9aea2f30c0c6408b P 99ac8d2d4cd2408bab694ed9594ef526: Bootstrap starting.
I20260924 00:56:44.735798 15990 tablet_bootstrap.cc:654] T 98480b849b974b3d9aea2f30c0c6408b P 99ac8d2d4cd2408bab694ed9594ef526: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:44.737155 15989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:44.720099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42208 (local address 127.8.64.129:44385)
0924 00:56:44.730086 (+ 9987us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:44.730095 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:44.730120 (+ 25us) server_negotiation.cc:407] Connection header received
0924 00:56:44.730206 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:44.730214 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:44.730350 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:44.730477 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:56:44.731394 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:44.732483 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:44.734909 (+ 2426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:44.736125 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:44.736277 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:44.736404 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:44.736717 (+ 313us) server_negotiation.cc:299] Negotiation successful
0924 00:56:44.736852 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9834,"thread_start_us":9726,"threads_started":1}
I20260924 00:56:44.737187 15988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:44.719591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:44385 (local address 127.0.0.1:42208)
0924 00:56:44.720182 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:44.720197 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:44.720307 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:44.730684 (+ 10377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:44.730692 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:44.730728 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:56:44.731189 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:44.731202 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:44.732621 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:44.732626 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:44.734727 (+ 2101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:44.734741 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:44.736344 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:44.736350 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:44.736563 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:44.736588 (+ 25us) client_negotiation.cc:769] Sending connection context
0924 00:56:44.736689 (+ 101us) client_negotiation.cc:240] Negotiation successful
0924 00:56:44.736854 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":624,"threads_started":1}
I20260924 00:56:44.737985 15543 tablet_service.cc:1467] Processing CreateTablet for tablet 98480b849b974b3d9aea2f30c0c6408b (DEFAULT_TABLE table=test-table [id=708efb5782c048a78ea0b0201ef30059]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:56:44.738380 15971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:44.718335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.130:44619 (local address 127.0.0.1:55896)
0924 00:56:44.718856 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:44.718872 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:44.718960 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:44.731944 (+ 12984us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:44.731948 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:44.731971 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:56:44.732187 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:44.732195 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:44.733528 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:44.733534 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:44.735552 (+ 2018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:44.735566 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:44.737630 (+ 2064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:44.737636 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:44.737838 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:44.737865 (+ 27us) client_negotiation.cc:769] Sending connection context
0924 00:56:44.737920 (+ 55us) client_negotiation.cc:240] Negotiation successful
0924 00:56:44.738018 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387}
I20260924 00:56:44.738493 15543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98480b849b974b3d9aea2f30c0c6408b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:44.738943 15990 tablet_bootstrap.cc:492] T 98480b849b974b3d9aea2f30c0c6408b P 99ac8d2d4cd2408bab694ed9594ef526: No bootstrap required, opened a new log
I20260924 00:56:44.739090 15990 ts_tablet_manager.cc:1397] T 98480b849b974b3d9aea2f30c0c6408b P 99ac8d2d4cd2408bab694ed9594ef526: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260924 00:56:44.739205 15986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:44.718272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55896 (local address 127.8.64.130:44619)
0924 00:56:44.731672 (+ 13400us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:44.731677 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:44.731692 (+ 15us) server_negotiation.cc:407] Connection header received
0924 00:56:44.731746 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:44.731749 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:44.731815 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:44.731929 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:56:44.732338 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:44.733367 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:44.735694 (+ 2327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:44.737349 (+ 1655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:44.738590 (+ 1241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:44.738690 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:44.738743 (+ 53us) server_negotiation.cc:299] Negotiation successful
0924 00:56:44.738878 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13299,"thread_start_us":13183,"threads_started":1}
I20260924 00:56:44.739850 15990 raft_consensus.cc:348] T 98480b849b974b3d9aea2f30c0c6408b P 99ac8d2d4cd2408bab694ed9594ef526 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72f23ce899264c60b7b9818e9cc2797c" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 44619 } } peers { permanent_uuid: "99ac8d2d4cd2408bab694ed9594ef526" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 33651 } } peers { permanent_uuid: "22c762dcd2e94b6f91721a9571613065" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 44385 } }
I20260924 00:56:44.740110 15990 raft_consensus.cc:374] T 98480b849b974b3d9aea2f30c0c6408b P 99ac8d2d4cd2408bab694ed9594ef526 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:44.740221 15990 raft_consensus.cc:729] T 98480b849b974b3d9aea2f30c0c6408b P 99ac8d2d4cd2408bab694ed9594ef526 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99ac8d2d4cd2408bab694ed9594ef526, State: Initialized, Role: FOLLOWER
I20260924 00:56:44.740494 15990 consensus_queue.cc:260] T 98480b849b974b3d9aea2f30c0c6408b P 99ac8d2d4cd2408bab694ed9594ef526 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72f23ce899264c60b7b9818e9cc2797c" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 44619 } } peers { permanent_uuid: "99ac8d2d4cd2408bab694ed9594ef526" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 33651 } } peers { permanent_uuid: "22c762dcd2e94b6f91721a9571613065" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 44385 } }
I20260924 00:56:44.742048 15990 ts_tablet_manager.cc:1428] T 98480b849b974b3d9aea2f30c0c6408b P 99ac8d2d4cd2408bab694ed9594ef526: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260924 00:56:44.745136 15691 tablet_service.cc:1467] Processing CreateTablet for tablet 98480b849b974b3d9aea2f30c0c6408b (DEFAULT_TABLE table=test-table [id=708efb5782c048a78ea0b0201ef30059]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:56:44.745503 15900 heartbeater.cc:503] Master 127.8.64.189:39347 was elected leader, sending a full tablet report...
I20260924 00:56:44.745589 15691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98480b849b974b3d9aea2f30c0c6408b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:44.754328 15995 tablet_bootstrap.cc:492] T 98480b849b974b3d9aea2f30c0c6408b P 72f23ce899264c60b7b9818e9cc2797c: Bootstrap starting.
I20260924 00:56:44.754689 15994 tablet_bootstrap.cc:492] T 98480b849b974b3d9aea2f30c0c6408b P 22c762dcd2e94b6f91721a9571613065: Bootstrap starting.
I20260924 00:56:44.756300 15995 tablet_bootstrap.cc:654] T 98480b849b974b3d9aea2f30c0c6408b P 72f23ce899264c60b7b9818e9cc2797c: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:44.756515 15994 tablet_bootstrap.cc:654] T 98480b849b974b3d9aea2f30c0c6408b P 22c762dcd2e94b6f91721a9571613065: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:44.758839 15995 tablet_bootstrap.cc:492] T 98480b849b974b3d9aea2f30c0c6408b P 72f23ce899264c60b7b9818e9cc2797c: No bootstrap required, opened a new log
I20260924 00:56:44.759213 15993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:44.741609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44743 (local address 127.8.64.131:37355)
0924 00:56:44.758766 (+ 17157us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:44.758903 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:44743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17042,"thread_start_us":16929,"threads_started":1}
I20260924 00:56:44.759235 15995 ts_tablet_manager.cc:1397] T 98480b849b974b3d9aea2f30c0c6408b P 72f23ce899264c60b7b9818e9cc2797c: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260924 00:56:44.760536 15993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:44.760149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44743 (local address 127.8.64.131:51067)
0924 00:56:44.760316 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:44.760402 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:44743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260924 00:56:44.760588 15994 tablet_bootstrap.cc:492] T 98480b849b974b3d9aea2f30c0c6408b P 22c762dcd2e94b6f91721a9571613065: No bootstrap required, opened a new log
I20260924 00:56:44.760459 15995 raft_consensus.cc:348] T 98480b849b974b3d9aea2f30c0c6408b P 72f23ce899264c60b7b9818e9cc2797c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72f23ce899264c60b7b9818e9cc2797c" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 44619 } } peers { permanent_uuid: "99ac8d2d4cd2408bab694ed9594ef526" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 33651 } } peers { permanent_uuid: "22c762dcd2e94b6f91721a9571613065" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 44385 } }
I20260924 00:56:44.760730 15995 raft_consensus.cc:374] T 98480b849b974b3d9aea2f30c0c6408b P 72f23ce899264c60b7b9818e9cc2797c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:44.760819 15995 raft_consensus.cc:729] T 98480b849b974b3d9aea2f30c0c6408b P 72f23ce899264c60b7b9818e9cc2797c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72f23ce899264c60b7b9818e9cc2797c, State: Initialized, Role: FOLLOWER
I20260924 00:56:44.760890 15994 ts_tablet_manager.cc:1397] T 98480b849b974b3d9aea2f30c0c6408b P 22c762dcd2e94b6f91721a9571613065: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260924 00:56:44.761068 15995 consensus_queue.cc:260] T 98480b849b974b3d9aea2f30c0c6408b P 72f23ce899264c60b7b9818e9cc2797c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72f23ce899264c60b7b9818e9cc2797c" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 44619 } } peers { permanent_uuid: "99ac8d2d4cd2408bab694ed9594ef526" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 33651 } } peers { permanent_uuid: "22c762dcd2e94b6f91721a9571613065" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 44385 } }
I20260924 00:56:44.761623 15994 raft_consensus.cc:348] T 98480b849b974b3d9aea2f30c0c6408b P 22c762dcd2e94b6f91721a9571613065 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72f23ce899264c60b7b9818e9cc2797c" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 44619 } } peers { permanent_uuid: "99ac8d2d4cd2408bab694ed9594ef526" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 33651 } } peers { permanent_uuid: "22c762dcd2e94b6f91721a9571613065" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 44385 } }
I20260924 00:56:44.761901 15994 raft_consensus.cc:374] T 98480b849b974b3d9aea2f30c0c6408b P 22c762dcd2e94b6f91721a9571613065 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:44.762017 15994 raft_consensus.cc:729] T 98480b849b974b3d9aea2f30c0c6408b P 22c762dcd2e94b6f91721a9571613065 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22c762dcd2e94b6f91721a9571613065, State: Initialized, Role: FOLLOWER
I20260924 00:56:44.762269 15994 consensus_queue.cc:260] T 98480b849b974b3d9aea2f30c0c6408b P 22c762dcd2e94b6f91721a9571613065 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72f23ce899264c60b7b9818e9cc2797c" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 44619 } } peers { permanent_uuid: "99ac8d2d4cd2408bab694ed9594ef526" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 33651 } } peers { permanent_uuid: "22c762dcd2e94b6f91721a9571613065" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 44385 } }
I20260924 00:56:44.763216 15999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:44.762135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44743 (local address 127.8.64.130:39205)
0924 00:56:44.762756 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:44.762899 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:44743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":326,"threads_started":1}
I20260924 00:56:44.764261 15994 ts_tablet_manager.cc:1428] T 98480b849b974b3d9aea2f30c0c6408b P 22c762dcd2e94b6f91721a9571613065: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260924 00:56:44.765055 15995 ts_tablet_manager.cc:1428] T 98480b849b974b3d9aea2f30c0c6408b P 72f23ce899264c60b7b9818e9cc2797c: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260924 00:56:44.766635 15759 heartbeater.cc:503] Master 127.8.64.189:39347 was elected leader, sending a full tablet report...
I20260924 00:56:44.767115 15610 heartbeater.cc:503] Master 127.8.64.189:39347 was elected leader, sending a full tablet report...
I20260924 00:56:44.770793 15999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:44.765695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44743 (local address 127.8.64.130:38367)
0924 00:56:44.770274 (+ 4579us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:44.770647 (+ 373us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:44743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4456}
I20260924 00:56:44.770846 16001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:44.764494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44743 (local address 127.8.64.129:33093)
0924 00:56:44.770426 (+ 5932us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:44.770564 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:44743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5765,"thread_start_us":5521,"threads_started":1}
I20260924 00:56:44.771833 16001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:44.771545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44743 (local address 127.8.64.129:57363)
0924 00:56:44.771672 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:44.771735 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:44743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260924 00:56:44.883737 15998 raft_consensus.cc:482] T 98480b849b974b3d9aea2f30c0c6408b P 72f23ce899264c60b7b9818e9cc2797c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:56:44.883941 15998 raft_consensus.cc:504] T 98480b849b974b3d9aea2f30c0c6408b P 72f23ce899264c60b7b9818e9cc2797c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72f23ce899264c60b7b9818e9cc2797c" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 44619 } } peers { permanent_uuid: "99ac8d2d4cd2408bab694ed9594ef526" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 33651 } } peers { permanent_uuid: "22c762dcd2e94b6f91721a9571613065" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 44385 } }
I20260924 00:56:44.885495 15998 leader_election.cc:288] T 98480b849b974b3d9aea2f30c0c6408b P 72f23ce899264c60b7b9818e9cc2797c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99ac8d2d4cd2408bab694ed9594ef526 (127.8.64.131:33651), 22c762dcd2e94b6f91721a9571613065 (127.8.64.129:44385)
I20260924 00:56:44.891999 15999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:44.885808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.131:33651 (local address 127.8.64.130:40063)
0924 00:56:44.886193 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:44.886245 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:44.886453 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:44.886862 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:44.886867 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:44.886896 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:56:44.887238 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:44.887258 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:44.888431 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:44.888436 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:44.890039 (+ 1603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:44.890054 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:44.891230 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:44.891236 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:44.891432 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:44.891479 (+ 47us) client_negotiation.cc:769] Sending connection context
0924 00:56:44.891564 (+ 85us) client_negotiation.cc:240] Negotiation successful
0924 00:56:44.891666 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":5}
I20260924 00:56:44.892019 15987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:44.885897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:40063 (local address 127.8.64.131:33651)
0924 00:56:44.886077 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:44.886083 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:44.886361 (+ 278us) server_negotiation.cc:407] Connection header received
0924 00:56:44.886614 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:44.886621 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:44.886722 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:44.886828 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:56:44.887381 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:44.888308 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:44.890196 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:44.891047 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:44.891166 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:44.891218 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:44.891617 (+ 399us) server_negotiation.cc:299] Negotiation successful
0924 00:56:44.891759 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260924 00:56:44.892180 15989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:44.885896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:39381 (local address 127.8.64.129:44385)
0924 00:56:44.886094 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:44.886101 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:44.886751 (+ 650us) server_negotiation.cc:407] Connection header received
0924 00:56:44.886929 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:44.886936 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:44.887015 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:44.887109 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:56:44.887706 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:44.888537 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:44.890333 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:44.891217 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:44.891372 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:44.891442 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:44.891778 (+ 336us) server_negotiation.cc:299] Negotiation successful
0924 00:56:44.891910 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260924 00:56:44.892313 16002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:44.885843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:44385 (local address 127.8.64.130:39381)
0924 00:56:44.886634 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:44.886652 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:44.886788 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:44.887131 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:44.887134 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:44.887152 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:56:44.887555 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:44.887566 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:44.888671 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:44.888675 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:44.890101 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:44.890137 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:44.891431 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:44.891436 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:44.891612 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:44.891644 (+ 32us) client_negotiation.cc:769] Sending connection context
0924 00:56:44.891734 (+ 90us) client_negotiation.cc:240] Negotiation successful
0924 00:56:44.891844 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"mutex_wait_us":67,"thread_start_us":473,"threads_started":1}
I20260924 00:56:44.893009 15853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98480b849b974b3d9aea2f30c0c6408b" candidate_uuid: "72f23ce899264c60b7b9818e9cc2797c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99ac8d2d4cd2408bab694ed9594ef526" is_pre_election: true
I20260924 00:56:44.893270 15563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98480b849b974b3d9aea2f30c0c6408b" candidate_uuid: "72f23ce899264c60b7b9818e9cc2797c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22c762dcd2e94b6f91721a9571613065" is_pre_election: true
I20260924 00:56:44.893392 15853 raft_consensus.cc:2454] T 98480b849b974b3d9aea2f30c0c6408b P 99ac8d2d4cd2408bab694ed9594ef526 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72f23ce899264c60b7b9818e9cc2797c in term 0.
I20260924 00:56:44.893530 15563 raft_consensus.cc:2454] T 98480b849b974b3d9aea2f30c0c6408b P 22c762dcd2e94b6f91721a9571613065 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72f23ce899264c60b7b9818e9cc2797c in term 0.
I20260924 00:56:44.894018 15645 leader_election.cc:302] T 98480b849b974b3d9aea2f30c0c6408b P 72f23ce899264c60b7b9818e9cc2797c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72f23ce899264c60b7b9818e9cc2797c, 99ac8d2d4cd2408bab694ed9594ef526; no voters:
I20260924 00:56:44.894438 15998 raft_consensus.cc:2781] T 98480b849b974b3d9aea2f30c0c6408b P 72f23ce899264c60b7b9818e9cc2797c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:56:44.894562 15998 raft_consensus.cc:482] T 98480b849b974b3d9aea2f30c0c6408b P 72f23ce899264c60b7b9818e9cc2797c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:56:44.894657 15998 raft_consensus.cc:3037] T 98480b849b974b3d9aea2f30c0c6408b P 72f23ce899264c60b7b9818e9cc2797c [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:44.896729 15998 raft_consensus.cc:504] T 98480b849b974b3d9aea2f30c0c6408b P 72f23ce899264c60b7b9818e9cc2797c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72f23ce899264c60b7b9818e9cc2797c" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 44619 } } peers { permanent_uuid: "99ac8d2d4cd2408bab694ed9594ef526" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 33651 } } peers { permanent_uuid: "22c762dcd2e94b6f91721a9571613065" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 44385 } }
I20260924 00:56:44.897347 15998 leader_election.cc:288] T 98480b849b974b3d9aea2f30c0c6408b P 72f23ce899264c60b7b9818e9cc2797c [CANDIDATE]: Term 1 election: Requested vote from peers 99ac8d2d4cd2408bab694ed9594ef526 (127.8.64.131:33651), 22c762dcd2e94b6f91721a9571613065 (127.8.64.129:44385)
I20260924 00:56:44.897750 15853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98480b849b974b3d9aea2f30c0c6408b" candidate_uuid: "72f23ce899264c60b7b9818e9cc2797c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99ac8d2d4cd2408bab694ed9594ef526"
I20260924 00:56:44.897907 15563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98480b849b974b3d9aea2f30c0c6408b" candidate_uuid: "72f23ce899264c60b7b9818e9cc2797c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22c762dcd2e94b6f91721a9571613065"
I20260924 00:56:44.898116 15563 raft_consensus.cc:3037] T 98480b849b974b3d9aea2f30c0c6408b P 22c762dcd2e94b6f91721a9571613065 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:44.898099 15853 raft_consensus.cc:3037] T 98480b849b974b3d9aea2f30c0c6408b P 99ac8d2d4cd2408bab694ed9594ef526 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:44.900134 15563 raft_consensus.cc:2454] T 98480b849b974b3d9aea2f30c0c6408b P 22c762dcd2e94b6f91721a9571613065 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72f23ce899264c60b7b9818e9cc2797c in term 1.
I20260924 00:56:44.900235 15853 raft_consensus.cc:2454] T 98480b849b974b3d9aea2f30c0c6408b P 99ac8d2d4cd2408bab694ed9594ef526 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72f23ce899264c60b7b9818e9cc2797c in term 1.
I20260924 00:56:44.900558 15646 leader_election.cc:302] T 98480b849b974b3d9aea2f30c0c6408b P 72f23ce899264c60b7b9818e9cc2797c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22c762dcd2e94b6f91721a9571613065, 72f23ce899264c60b7b9818e9cc2797c; no voters:
I20260924 00:56:44.900884 15998 raft_consensus.cc:2781] T 98480b849b974b3d9aea2f30c0c6408b P 72f23ce899264c60b7b9818e9cc2797c [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:56:44.901335 15998 raft_consensus.cc:686] T 98480b849b974b3d9aea2f30c0c6408b P 72f23ce899264c60b7b9818e9cc2797c [term 1 LEADER]: Becoming Leader. State: Replica: 72f23ce899264c60b7b9818e9cc2797c, State: Running, Role: LEADER
I20260924 00:56:44.902385 16002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:44.901895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44743 (local address 127.8.64.130:51183)
0924 00:56:44.902065 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:44.902166 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:44743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260924 00:56:44.902251 15998 consensus_queue.cc:237] T 98480b849b974b3d9aea2f30c0c6408b P 72f23ce899264c60b7b9818e9cc2797c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72f23ce899264c60b7b9818e9cc2797c" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 44619 } } peers { permanent_uuid: "99ac8d2d4cd2408bab694ed9594ef526" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 33651 } } peers { permanent_uuid: "22c762dcd2e94b6f91721a9571613065" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 44385 } }
I20260924 00:56:44.906899 15239 catalog_manager.cc:5495] T 98480b849b974b3d9aea2f30c0c6408b P 72f23ce899264c60b7b9818e9cc2797c reported cstate change: term changed from 0 to 1, leader changed from <none> to 72f23ce899264c60b7b9818e9cc2797c (127.8.64.130). New cstate: current_term: 1 leader_uuid: "72f23ce899264c60b7b9818e9cc2797c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72f23ce899264c60b7b9818e9cc2797c" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 44619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99ac8d2d4cd2408bab694ed9594ef526" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 33651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22c762dcd2e94b6f91721a9571613065" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 44385 } health_report { overall_health: UNKNOWN } } }
I20260924 00:56:44.969614 15986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:44.963373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55908 (local address 127.8.64.130:44619)
0924 00:56:44.963588 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:44.963593 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:44.963610 (+ 17us) server_negotiation.cc:407] Connection header received
0924 00:56:44.963823 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:44.963827 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:44.963900 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:44.964028 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:56:44.964445 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:44.965350 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:44.967232 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:44.967746 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:44.967920 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:44.968392 (+ 472us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:56:44.968941 (+ 549us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:56:44.969056 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:44.969272 (+ 216us) server_negotiation.cc:299] Negotiation successful
0924 00:56:44.969357 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260924 00:56:44.973924 15563 raft_consensus.cc:1261] T 98480b849b974b3d9aea2f30c0c6408b P 22c762dcd2e94b6f91721a9571613065 [term 1 FOLLOWER]: Refusing update from remote peer 72f23ce899264c60b7b9818e9cc2797c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:56:44.974264 15853 raft_consensus.cc:1261] T 98480b849b974b3d9aea2f30c0c6408b P 99ac8d2d4cd2408bab694ed9594ef526 [term 1 FOLLOWER]: Refusing update from remote peer 72f23ce899264c60b7b9818e9cc2797c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:56:44.974663 16003 consensus_queue.cc:1035] T 98480b849b974b3d9aea2f30c0c6408b P 72f23ce899264c60b7b9818e9cc2797c [LEADER]: Connected to new peer: Peer: permanent_uuid: "22c762dcd2e94b6f91721a9571613065" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 44385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:56:44.975080 15998 consensus_queue.cc:1035] T 98480b849b974b3d9aea2f30c0c6408b P 72f23ce899264c60b7b9818e9cc2797c [LEADER]: Connected to new peer: Peer: permanent_uuid: "99ac8d2d4cd2408bab694ed9594ef526" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 33651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:56:44.978217 15993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:44.977635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44743 (local address 127.8.64.131:56387)
0924 00:56:44.977902 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:44.978041 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:44743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
I20260924 00:56:44.979342 16001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:44.977195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44743 (local address 127.8.64.129:44027)
0924 00:56:44.978979 (+ 1784us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:44.979110 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:44743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1621}
I20260924 00:56:44.984815 16001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:44.984227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44743 (local address 127.8.64.129:38027)
0924 00:56:44.984506 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:44.984611 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:44743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144}
I20260924 00:56:44.986371 16002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:44.985746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44743 (local address 127.8.64.130:49739)
0924 00:56:44.986033 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:44.986154 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:44743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179}
I20260924 00:56:44.995972 15239 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37426:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:56:44.996347 15239 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37426:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:56:45.000397 16002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:44.999687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44743 (local address 127.8.64.130:41779)
0924 00:56:45.000117 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:45.000207 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:44743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319}
I20260924 00:56:45.001493 15993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:45.001075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44743 (local address 127.8.64.131:38979)
0924 00:56:45.001230 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:45.001317 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:44743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260924 00:56:45.006989 15239 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1d362bb4f4dd41adaee4592166347b34: Sending DeleteTablet for 3 replicas of tablet 98480b849b974b3d9aea2f30c0c6408b
I20260924 00:56:45.007916 15833 tablet_service.cc:1514] Processing DeleteTablet for tablet 98480b849b974b3d9aea2f30c0c6408b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:56:44 UTC) from {username='oryx', principal='oryx/127.8.64.189@KRBTEST.COM'} at 127.0.0.1:35896
I20260924 00:56:45.008229 15691 tablet_service.cc:1514] Processing DeleteTablet for tablet 98480b849b974b3d9aea2f30c0c6408b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:56:44 UTC) from {username='oryx', principal='oryx/127.8.64.189@KRBTEST.COM'} at 127.0.0.1:55896
I20260924 00:56:45.008512 15832 tablet_service.cc:1514] Processing DeleteTablet for tablet 98480b849b974b3d9aea2f30c0c6408b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:56:44 UTC) from {username='oryx', principal='oryx/127.8.64.189@KRBTEST.COM'} at 127.0.0.1:35896
I20260924 00:56:45.010775 15543 tablet_service.cc:1514] Processing DeleteTablet for tablet 98480b849b974b3d9aea2f30c0c6408b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:56:44 UTC) from {username='oryx', principal='oryx/127.8.64.189@KRBTEST.COM'} at 127.0.0.1:42208
I20260924 00:56:45.011214 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 15424
I20260924 00:56:45.023058 16017 tablet_replica.cc:330] T 98480b849b974b3d9aea2f30c0c6408b P 72f23ce899264c60b7b9818e9cc2797c: stopping tablet replica
I20260924 00:56:45.023511 16017 raft_consensus.cc:2229] T 98480b849b974b3d9aea2f30c0c6408b P 72f23ce899264c60b7b9818e9cc2797c [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:56:45.024456 16017 raft_consensus.cc:2258] T 98480b849b974b3d9aea2f30c0c6408b P 72f23ce899264c60b7b9818e9cc2797c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:56:45.025230 16016 tablet_replica.cc:330] T 98480b849b974b3d9aea2f30c0c6408b P 99ac8d2d4cd2408bab694ed9594ef526: stopping tablet replica
I20260924 00:56:45.025445 16016 raft_consensus.cc:2229] T 98480b849b974b3d9aea2f30c0c6408b P 99ac8d2d4cd2408bab694ed9594ef526 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:56:45.025738 16016 raft_consensus.cc:2258] T 98480b849b974b3d9aea2f30c0c6408b P 99ac8d2d4cd2408bab694ed9594ef526 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:56:45.026862 16017 ts_tablet_manager.cc:1905] T 98480b849b974b3d9aea2f30c0c6408b P 72f23ce899264c60b7b9818e9cc2797c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:56:45.027417 16018 tablet_replica.cc:330] T 98480b849b974b3d9aea2f30c0c6408b P 22c762dcd2e94b6f91721a9571613065: stopping tablet replica
I20260924 00:56:45.027655 16018 raft_consensus.cc:2229] T 98480b849b974b3d9aea2f30c0c6408b P 22c762dcd2e94b6f91721a9571613065 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:56:45.028002 16018 raft_consensus.cc:2258] T 98480b849b974b3d9aea2f30c0c6408b P 22c762dcd2e94b6f91721a9571613065 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:56:45.029016 16018 ts_tablet_manager.cc:1905] T 98480b849b974b3d9aea2f30c0c6408b P 22c762dcd2e94b6f91721a9571613065: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:56:45.034384 16017 ts_tablet_manager.cc:1918] T 98480b849b974b3d9aea2f30c0c6408b P 72f23ce899264c60b7b9818e9cc2797c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:56:45.035076 16016 ts_tablet_manager.cc:1905] T 98480b849b974b3d9aea2f30c0c6408b P 99ac8d2d4cd2408bab694ed9594ef526: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:56:45.036042 16018 ts_tablet_manager.cc:1918] T 98480b849b974b3d9aea2f30c0c6408b P 22c762dcd2e94b6f91721a9571613065: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:56:45.036201 16018 log.cc:1196] T 98480b849b974b3d9aea2f30c0c6408b P 22c762dcd2e94b6f91721a9571613065: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-0/wal/wals/98480b849b974b3d9aea2f30c0c6408b
I20260924 00:56:45.036667 16018 ts_tablet_manager.cc:1939] T 98480b849b974b3d9aea2f30c0c6408b P 22c762dcd2e94b6f91721a9571613065: Deleting consensus metadata
I20260924 00:56:45.037920 15226 catalog_manager.cc:4841] TS 22c762dcd2e94b6f91721a9571613065 (127.8.64.129:44385): tablet 98480b849b974b3d9aea2f30c0c6408b (table test-table [id=708efb5782c048a78ea0b0201ef30059]) successfully deleted
I20260924 00:56:45.042145 16016 ts_tablet_manager.cc:1918] T 98480b849b974b3d9aea2f30c0c6408b P 99ac8d2d4cd2408bab694ed9594ef526: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:56:45.042290 16016 log.cc:1196] T 98480b849b974b3d9aea2f30c0c6408b P 99ac8d2d4cd2408bab694ed9594ef526: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-2/wal/wals/98480b849b974b3d9aea2f30c0c6408b
I20260924 00:56:45.042706 16016 ts_tablet_manager.cc:1939] T 98480b849b974b3d9aea2f30c0c6408b P 99ac8d2d4cd2408bab694ed9594ef526: Deleting consensus metadata
I20260924 00:56:45.043788 16017 log.cc:1196] T 98480b849b974b3d9aea2f30c0c6408b P 72f23ce899264c60b7b9818e9cc2797c: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790211343099518-8450-0/minicluster-data/ts-1/wal/wals/98480b849b974b3d9aea2f30c0c6408b
I20260924 00:56:45.043900 15225 catalog_manager.cc:4841] TS 99ac8d2d4cd2408bab694ed9594ef526 (127.8.64.131:33651): tablet 98480b849b974b3d9aea2f30c0c6408b (table test-table [id=708efb5782c048a78ea0b0201ef30059]) successfully deleted
I20260924 00:56:45.044405 16017 ts_tablet_manager.cc:1939] T 98480b849b974b3d9aea2f30c0c6408b P 72f23ce899264c60b7b9818e9cc2797c: Deleting consensus metadata
I20260924 00:56:45.045171 15989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:45.012351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42220 (local address 127.8.64.129:44385)
0924 00:56:45.012519 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:45.012525 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:45.026808 (+ 14283us) server_negotiation.cc:407] Connection header received
0924 00:56:45.027014 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:45.027020 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:45.027118 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:45.027244 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:45.029456 (+ 2212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:45.030657 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:45.032586 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:45.033286 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:45.033401 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:45.037123 (+ 3722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:45.037142 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:45.037162 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:45.037237 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:45.041435 (+ 4198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:45.042361 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:45.042370 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:45.042374 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:45.042550 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:45.042989 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:45.042994 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:45.042997 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:45.043443 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:45.043741 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:45.044401 (+ 660us) server_negotiation.cc:299] Negotiation successful
0924 00:56:45.044802 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260924 00:56:45.045513 15226 catalog_manager.cc:4841] TS 72f23ce899264c60b7b9818e9cc2797c (127.8.64.130:44619): tablet 98480b849b974b3d9aea2f30c0c6408b (table test-table [id=708efb5782c048a78ea0b0201ef30059]) successfully deleted
I20260924 00:56:45.046067 15603 generic_service.cc:196] Checking for leaks (request via RPC)
W20260924 00:56:45.046209 15225 catalog_manager.cc:4808] TS 99ac8d2d4cd2408bab694ed9594ef526 (127.8.64.131:33651): delete failed for tablet 98480b849b974b3d9aea2f30c0c6408b because the tablet was not found. No further retry: Not found: Tablet not found: 98480b849b974b3d9aea2f30c0c6408b
I20260924 00:56:45.177265 15971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:45.176854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44743 (local address 127.0.0.1:53020)
0924 00:56:45.177034 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:45.177129 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:44743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260924 00:56:45.178345 15971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:45.177952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:44743 (local address 127.0.0.1:53036)
0924 00:56:45.178119 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:45.178194 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:44743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260924 00:56:45.204540 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 15424
I20260924 00:56:45.228768 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 15621
I20260924 00:56:45.244967 15986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:45.229554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55922 (local address 127.8.64.130:44619)
0924 00:56:45.229726 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:45.229731 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:45.229807 (+ 76us) server_negotiation.cc:407] Connection header received
0924 00:56:45.230008 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:45.230011 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:45.230070 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:45.230157 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:45.231874 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:45.232691 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:45.234586 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:45.235000 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:45.235099 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:45.238412 (+ 3313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:45.238430 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:45.238448 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:45.238479 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:45.241806 (+ 3327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:45.242761 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:45.242768 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:45.242770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:45.242922 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:45.243336 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:45.243339 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:45.243341 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:45.243767 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:45.244015 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:45.244347 (+ 332us) server_negotiation.cc:299] Negotiation successful
0924 00:56:45.244524 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260924 00:56:45.245652 15751 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:56:45.364894 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 15621
I20260924 00:56:45.397701 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 15770
I20260924 00:56:45.414140 16022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:45.398870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35902 (local address 127.8.64.131:33651)
0924 00:56:45.399327 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:45.399331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:45.399348 (+ 17us) server_negotiation.cc:407] Connection header received
0924 00:56:45.399408 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:45.399411 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:45.399498 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:45.401815 (+ 2317us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:45.401938 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:45.403026 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:45.405160 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:45.405676 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:45.405781 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:45.408406 (+ 2625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:45.408420 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:45.408436 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:45.408466 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:45.411450 (+ 2984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:45.412164 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:45.412171 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:45.412173 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:45.412295 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:45.412685 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:45.412688 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:45.412690 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:45.413036 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:45.413479 (+ 443us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:45.413590 (+ 111us) server_negotiation.cc:299] Negotiation successful
0924 00:56:45.413751 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":264,"threads_started":1}
I20260924 00:56:45.414856 15893 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:56:45.538802 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 15770
I20260924 00:56:45.561223 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 15099
I20260924 00:56:45.562340 15270 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:56:45.681491 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 15099
I20260924 00:56:45.699630 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 15204
I20260924 00:56:45.700697 15459 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:56:45.812553 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 15204
2026-09-24T00:56:45Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (9915 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:56:45 dist-test-slave-mmhs krb5kdc[16029](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:56:45 dist-test-slave-mmhs krb5kdc[16029](info): set up 2 sockets
Sep 24 00:56:45 dist-test-slave-mmhs krb5kdc[16029](info): commencing operation
krb5kdc: starting...
W20260924 00:56:46.908949 8450 mini_kdc.cc:121] Time spent starting KDC: real 1.051s user 0.004s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Sep 24 00:56:47 dist-test-slave-mmhs krb5kdc[16029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:56:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:56:47Z Disabled control of system clock
WARNING: no policy specified for oryx/127.8.64.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.64.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.64.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.190.keytab.
Entry for principal oryx/127.8.64.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.190.keytab.
I20260924 00:56:47.134192 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:36845
--webserver_interface=127.8.64.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.190.keytab
--principal=oryx/127.8.64.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:39031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:36845
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:56:47.453574 16042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:47.453898 16042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:47.454066 16042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:47.461238 16042 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:56:47.461380 16042 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260924 00:56:47.461472 16042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:47.461582 16042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:47.461673 16042 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:56:47.461758 16042 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:56:47.471524 16042 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:39031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:36845
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.190.keytab
--principal=oryx/127.8.64.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:36845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:47.472788 16042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:47.474107 16042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:47.485695 16047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:47.485728 16050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:47.485766 16048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:47.486779 16042 server_base.cc:1034] running on GCE node
I20260924 00:56:47.487470 16042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:47.488688 16042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:47.489928 16042 hybrid_clock.cc:648] HybridClock initialized: now 1790211407489873 us; error 29 us; skew 500 ppm
I20260924 00:56:47.490221 16042 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:47 dist-test-slave-mmhs krb5kdc[16029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211407, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.64.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:47.493244 16042 init.cc:377] Logged in from keytab as oryx/127.8.64.190@KRBTEST.COM (short username oryx)
I20260924 00:56:47.494686 16042 webserver.cc:460] Webserver started at http://127.8.64.190:46175/ using document root <none> and password file <none>
I20260924 00:56:47.495323 16042 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:47.495414 16042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:47.495669 16042 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:56:47.498562 16042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "debc27d99cb84de7940c1a40cfe8fff2"
format_stamp: "Formatted at 2026-09-24 00:56:47 on dist-test-slave-mmhs"
server_key: "b61b2a97a7bf01b817990695aef32546"
server_key_iv: "b4034c12741374bb03a3b337abdb77c8"
server_key_version: "encryptionkey@0"
I20260924 00:56:47.499292 16042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "debc27d99cb84de7940c1a40cfe8fff2"
format_stamp: "Formatted at 2026-09-24 00:56:47 on dist-test-slave-mmhs"
server_key: "b61b2a97a7bf01b817990695aef32546"
server_key_iv: "b4034c12741374bb03a3b337abdb77c8"
server_key_version: "encryptionkey@0"
I20260924 00:56:47.504222 16042 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260924 00:56:47.507517 16057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:47.508837 16042 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260924 00:56:47.509008 16042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "debc27d99cb84de7940c1a40cfe8fff2"
format_stamp: "Formatted at 2026-09-24 00:56:47 on dist-test-slave-mmhs"
server_key: "b61b2a97a7bf01b817990695aef32546"
server_key_iv: "b4034c12741374bb03a3b337abdb77c8"
server_key_version: "encryptionkey@0"
I20260924 00:56:47.509235 16042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:47.532610 16042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:47.536800 16042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:47.537091 16042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:47.558945 16042 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.190:36845
I20260924 00:56:47.558964 16118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.190:36845 every 8 connection(s)
I20260924 00:56:47.560401 16042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 00:56:47.564201 16119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:47.565003 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 16042
I20260924 00:56:47.565197 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 00:56:47.565832 8450 external_mini_cluster.cc:1428] Setting key 9c3100bd8d952b923db32cbf84d90f6c
I20260924 00:56:47.577945 16119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P debc27d99cb84de7940c1a40cfe8fff2: Bootstrap starting.
Sep 24 00:56:47 dist-test-slave-mmhs krb5kdc[16029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM
I20260924 00:56:47.581496 16119 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P debc27d99cb84de7940c1a40cfe8fff2: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:47.582769 16119 log.cc:824] T 00000000000000000000000000000000 P debc27d99cb84de7940c1a40cfe8fff2: Log is configured to *not* fsync() on all Append() calls
I20260924 00:56:47.585961 16119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P debc27d99cb84de7940c1a40cfe8fff2: No bootstrap required, opened a new log
I20260924 00:56:47.588562 16122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:47.567823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34576 (local address 127.8.64.190:36845)
0924 00:56:47.568616 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:47.568626 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:47.568668 (+ 42us) server_negotiation.cc:407] Connection header received
0924 00:56:47.570238 (+ 1570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:47.570259 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:47.570716 (+ 457us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:47.571042 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:47.572749 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:47.573993 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:47.575784 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:47.576420 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:47.576543 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:47.580920 (+ 4377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:47.580927 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:47.580941 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:47.580973 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:47.584547 (+ 3574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:47.585225 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:47.585238 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:47.585245 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:47.585412 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:47.585784 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:47.585790 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:47.585793 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:47.586405 (+ 612us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:47.586707 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:47.587193 (+ 486us) server_negotiation.cc:299] Negotiation successful
0924 00:56:47.587595 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":347,"threads_started":1}
I20260924 00:56:47.593847 16119 raft_consensus.cc:348] T 00000000000000000000000000000000 P debc27d99cb84de7940c1a40cfe8fff2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "debc27d99cb84de7940c1a40cfe8fff2" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 36845 } }
I20260924 00:56:47.594194 16119 raft_consensus.cc:374] T 00000000000000000000000000000000 P debc27d99cb84de7940c1a40cfe8fff2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:47.594334 16119 raft_consensus.cc:729] T 00000000000000000000000000000000 P debc27d99cb84de7940c1a40cfe8fff2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: debc27d99cb84de7940c1a40cfe8fff2, State: Initialized, Role: FOLLOWER
I20260924 00:56:47.594954 16119 consensus_queue.cc:260] T 00000000000000000000000000000000 P debc27d99cb84de7940c1a40cfe8fff2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "debc27d99cb84de7940c1a40cfe8fff2" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 36845 } }
I20260924 00:56:47.595207 16119 raft_consensus.cc:388] T 00000000000000000000000000000000 P debc27d99cb84de7940c1a40cfe8fff2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:56:47.595347 16119 raft_consensus.cc:482] T 00000000000000000000000000000000 P debc27d99cb84de7940c1a40cfe8fff2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:56:47.595541 16119 raft_consensus.cc:3037] T 00000000000000000000000000000000 P debc27d99cb84de7940c1a40cfe8fff2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:47.597620 16119 raft_consensus.cc:504] T 00000000000000000000000000000000 P debc27d99cb84de7940c1a40cfe8fff2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "debc27d99cb84de7940c1a40cfe8fff2" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 36845 } }
I20260924 00:56:47.598102 16119 leader_election.cc:302] T 00000000000000000000000000000000 P debc27d99cb84de7940c1a40cfe8fff2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: debc27d99cb84de7940c1a40cfe8fff2; no voters:
I20260924 00:56:47.598795 16119 leader_election.cc:288] T 00000000000000000000000000000000 P debc27d99cb84de7940c1a40cfe8fff2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:56:47.598883 16124 raft_consensus.cc:2781] T 00000000000000000000000000000000 P debc27d99cb84de7940c1a40cfe8fff2 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:56:47.599714 16124 raft_consensus.cc:686] T 00000000000000000000000000000000 P debc27d99cb84de7940c1a40cfe8fff2 [term 1 LEADER]: Becoming Leader. State: Replica: debc27d99cb84de7940c1a40cfe8fff2, State: Running, Role: LEADER
I20260924 00:56:47.600255 16124 consensus_queue.cc:237] T 00000000000000000000000000000000 P debc27d99cb84de7940c1a40cfe8fff2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "debc27d99cb84de7940c1a40cfe8fff2" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 36845 } }
I20260924 00:56:47.600982 16119 sys_catalog.cc:564] T 00000000000000000000000000000000 P debc27d99cb84de7940c1a40cfe8fff2 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:56:47.603044 16126 sys_catalog.cc:455] T 00000000000000000000000000000000 P debc27d99cb84de7940c1a40cfe8fff2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader debc27d99cb84de7940c1a40cfe8fff2. Latest consensus state: current_term: 1 leader_uuid: "debc27d99cb84de7940c1a40cfe8fff2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "debc27d99cb84de7940c1a40cfe8fff2" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 36845 } } }
I20260924 00:56:47.603505 16126 sys_catalog.cc:458] T 00000000000000000000000000000000 P debc27d99cb84de7940c1a40cfe8fff2 [sys.catalog]: This master's current role is: LEADER
I20260924 00:56:47.604077 16125 sys_catalog.cc:455] T 00000000000000000000000000000000 P debc27d99cb84de7940c1a40cfe8fff2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "debc27d99cb84de7940c1a40cfe8fff2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "debc27d99cb84de7940c1a40cfe8fff2" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 36845 } } }
I20260924 00:56:47.604424 16125 sys_catalog.cc:458] T 00000000000000000000000000000000 P debc27d99cb84de7940c1a40cfe8fff2 [sys.catalog]: This master's current role is: LEADER
I20260924 00:56:47.604952 16132 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:56:47.613729 16132 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:56:47.622189 16132 catalog_manager.cc:1347] Generated new cluster ID: 4dc2ab0cee2d488f88aded96648dc4c7
I20260924 00:56:47.622320 16132 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:56:47.652127 16132 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:56:47.653455 16132 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:56:47.664644 16132 catalog_manager.cc:5868] T 00000000000000000000000000000000 P debc27d99cb84de7940c1a40cfe8fff2: Generated new TSK 0
I20260924 00:56:47.665496 16132 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.8.64.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.64.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.64.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.129.keytab.
Entry for principal oryx/127.8.64.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.129.keytab.
I20260924 00:56:47.730723 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.129:0
--local_ip_for_outbound_sockets=127.8.64.129
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:36845
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.129.keytab
--principal=oryx/127.8.64.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:39031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:56:48.072395 16145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:48.072666 16145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:48.072762 16145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:48.079748 16145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:48.079852 16145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:48.079933 16145 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 00:56:48.080016 16145 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.129
I20260924 00:56:48.089612 16145 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:39031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.129.keytab
--principal=oryx/127.8.64.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.64.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.64.190:36845
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.129
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:48.090869 16145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:48.092195 16145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:48.105276 16151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:48.105322 16153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:48.105957 16145 server_base.cc:1034] running on GCE node
W20260924 00:56:48.106032 16150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:48.106817 16145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:48.107867 16145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:48.109105 16145 hybrid_clock.cc:648] HybridClock initialized: now 1790211408109040 us; error 40 us; skew 500 ppm
I20260924 00:56:48.109405 16145 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:48 dist-test-slave-mmhs krb5kdc[16029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211408, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.64.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:48.112519 16145 init.cc:377] Logged in from keytab as oryx/127.8.64.129@KRBTEST.COM (short username oryx)
I20260924 00:56:48.113958 16145 webserver.cc:460] Webserver started at http://127.8.64.129:39291/ using document root <none> and password file <none>
I20260924 00:56:48.114702 16145 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:48.114831 16145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:48.115147 16145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:56:48.118086 16145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-0/data/instance:
uuid: "adfab66f8ae74a76ac0d1d09df6a617c"
format_stamp: "Formatted at 2026-09-24 00:56:48 on dist-test-slave-mmhs"
server_key: "97705de79e14ecf0b6606ce428483949"
server_key_iv: "a6bf3d243c5da3fc87d2e9e3b142826c"
server_key_version: "encryptionkey@0"
I20260924 00:56:48.118917 16145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance:
uuid: "adfab66f8ae74a76ac0d1d09df6a617c"
format_stamp: "Formatted at 2026-09-24 00:56:48 on dist-test-slave-mmhs"
server_key: "97705de79e14ecf0b6606ce428483949"
server_key_iv: "a6bf3d243c5da3fc87d2e9e3b142826c"
server_key_version: "encryptionkey@0"
I20260924 00:56:48.123737 16145 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260924 00:56:48.127029 16160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:48.128299 16145 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260924 00:56:48.128472 16145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-0/wal
uuid: "adfab66f8ae74a76ac0d1d09df6a617c"
format_stamp: "Formatted at 2026-09-24 00:56:48 on dist-test-slave-mmhs"
server_key: "97705de79e14ecf0b6606ce428483949"
server_key_iv: "a6bf3d243c5da3fc87d2e9e3b142826c"
server_key_version: "encryptionkey@0"
I20260924 00:56:48.128682 16145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:48.151275 16145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:48.155239 16145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:48.155520 16145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:48.158070 16145 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:56:48.158198 16145 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:48.158362 16145 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:56:48.158437 16145 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 00:56:48 dist-test-slave-mmhs krb5kdc[16029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211408, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.64.129@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM
I20260924 00:56:48.182255 16169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:48.158875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:60493 (local address 127.8.64.190:36845)
0924 00:56:48.159272 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:48.159277 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:48.161009 (+ 1732us) server_negotiation.cc:407] Connection header received
0924 00:56:48.163285 (+ 2276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:48.163290 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:48.163392 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:48.163511 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:48.166208 (+ 2697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:48.167202 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:48.169600 (+ 2398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:48.170139 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:48.170272 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:48.175146 (+ 4874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:48.175154 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:48.175161 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:48.175202 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:48.178703 (+ 3501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:48.179662 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:48.179667 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:48.179672 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:48.179823 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:48.180541 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:48.180547 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:48.180551 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:48.180840 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:48.180991 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:48.181691 (+ 700us) server_negotiation.cc:299] Negotiation successful
0924 00:56:48.181903 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":203,"threads_started":1}
I20260924 00:56:48.183257 16170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:48.160023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:36845 (local address 127.8.64.129:60493)
0924 00:56:48.160852 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:48.160891 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:48.163008 (+ 2117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:48.163806 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:48.163820 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:48.164809 (+ 989us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:48.165911 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:48.165952 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:48.167395 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:48.167398 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:48.169461 (+ 2063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:48.169473 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:48.170609 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:48.170622 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:48.170919 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:48.171639 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:48.171677 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:48.174838 (+ 3161us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:48.178848 (+ 4010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:48.178858 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:48.178877 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:48.179507 (+ 630us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:48.179961 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:48.179969 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:48.179979 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:48.180395 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:48.180985 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:48.180993 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:48.181337 (+ 344us) client_negotiation.cc:769] Sending connection context
0924 00:56:48.181601 (+ 264us) client_negotiation.cc:240] Negotiation successful
0924 00:56:48.182113 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":359,"threads_started":1}
I20260924 00:56:48.204136 16145 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.129:35697
I20260924 00:56:48.204138 16280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.129:35697 every 8 connection(s)
I20260924 00:56:48.205653 16145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
I20260924 00:56:48.207892 16281 heartbeater.cc:347] Connected to a master server at 127.8.64.190:36845
I20260924 00:56:48.208187 16281 heartbeater.cc:464] Registering TS with master...
I20260924 00:56:48.208863 16281 heartbeater.cc:511] Master 127.8.64.190:36845 requested a full tablet report, sending...
I20260924 00:56:48.211181 16074 ts_manager.cc:194] Registered new tserver with Master: adfab66f8ae74a76ac0d1d09df6a617c (127.8.64.129:35697)
I20260924 00:56:48.212951 16074 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.64.129@KRBTEST.COM'} at 127.8.64.129:60493
I20260924 00:56:48.215643 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 16145
I20260924 00:56:48.215830 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance
I20260924 00:56:48.216136 16169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:48.197685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:35719 (local address 127.8.64.190:36845)
0924 00:56:48.197922 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:48.197931 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:48.203029 (+ 5098us) server_negotiation.cc:407] Connection header received
0924 00:56:48.203296 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:48.203304 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:48.203421 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:48.203517 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:48.205093 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:48.205829 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:48.207766 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:48.208329 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:48.208471 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:48.210458 (+ 1987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:48.210463 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:48.210466 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:48.210495 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:48.213424 (+ 2929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:48.214192 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:48.214197 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:48.214200 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:48.214381 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:48.214747 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:48.214752 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:48.214756 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:48.215018 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:48.215188 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:48.215561 (+ 373us) server_negotiation.cc:299] Negotiation successful
0924 00:56:48.215771 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260924 00:56:48.216236 16265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:48.197311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:36845 (local address 127.8.64.129:35719)
0924 00:56:48.202879 (+ 5568us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:48.202908 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:48.203085 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:48.203757 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:48.203765 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:48.204547 (+ 782us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:48.204913 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:48.204948 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:48.205968 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:48.205974 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:48.207639 (+ 1665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:48.207651 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:48.208522 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:48.208529 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:48.208705 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:48.209369 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:48.209393 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:48.210217 (+ 824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:48.213551 (+ 3334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:48.213557 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:48.213561 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:48.214069 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:48.214488 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:48.214493 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:48.214500 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:48.214636 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:48.215154 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:48.215160 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:48.215331 (+ 171us) client_negotiation.cc:769] Sending connection context
0924 00:56:48.215464 (+ 133us) client_negotiation.cc:240] Negotiation successful
0924 00:56:48.215779 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5370,"thread_start_us":5229,"threads_started":1}
I20260924 00:56:48.216630 8450 external_mini_cluster.cc:1428] Setting key bd5a77cdb43ec6da9c4a46ce02621363
WARNING: no policy specified for oryx/127.8.64.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.64.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.64.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.130.keytab.
Entry for principal oryx/127.8.64.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.130.keytab.
I20260924 00:56:48.265350 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.130:0
--local_ip_for_outbound_sockets=127.8.64.130
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:36845
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.130.keytab
--principal=oryx/127.8.64.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:39031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:56:48.599314 16285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:48.599574 16285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:48.599738 16285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:48.606973 16285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:48.607154 16285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:48.607327 16285 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 00:56:48.607447 16285 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.130
I20260924 00:56:48.617180 16285 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:39031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.130.keytab
--principal=oryx/127.8.64.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.64.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.64.190:36845
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.130
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:48.618498 16285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:48.619908 16285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:48.633068 16290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:48.633104 16291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:48.633040 16293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:48.634260 16285 server_base.cc:1034] running on GCE node
I20260924 00:56:48.634943 16285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:48.635846 16285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:48.637107 16285 hybrid_clock.cc:648] HybridClock initialized: now 1790211408637053 us; error 60 us; skew 500 ppm
I20260924 00:56:48.637411 16285 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:48 dist-test-slave-mmhs krb5kdc[16029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211408, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.64.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:48.640540 16285 init.cc:377] Logged in from keytab as oryx/127.8.64.130@KRBTEST.COM (short username oryx)
I20260924 00:56:48.641999 16285 webserver.cc:460] Webserver started at http://127.8.64.130:38891/ using document root <none> and password file <none>
I20260924 00:56:48.642752 16285 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:48.642896 16285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:48.643221 16285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:56:48.646159 16285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-1/data/instance:
uuid: "456e100a017b491d8d11fd3a68031fd6"
format_stamp: "Formatted at 2026-09-24 00:56:48 on dist-test-slave-mmhs"
server_key: "c3f7ba4f62cbf47050cd12bda9d09233"
server_key_iv: "f1035ad4607cb544192f1cd054df55fa"
server_key_version: "encryptionkey@0"
I20260924 00:56:48.647063 16285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance:
uuid: "456e100a017b491d8d11fd3a68031fd6"
format_stamp: "Formatted at 2026-09-24 00:56:48 on dist-test-slave-mmhs"
server_key: "c3f7ba4f62cbf47050cd12bda9d09233"
server_key_iv: "f1035ad4607cb544192f1cd054df55fa"
server_key_version: "encryptionkey@0"
I20260924 00:56:48.652145 16285 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260924 00:56:48.655526 16300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:48.657006 16285 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260924 00:56:48.657182 16285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-1/wal
uuid: "456e100a017b491d8d11fd3a68031fd6"
format_stamp: "Formatted at 2026-09-24 00:56:48 on dist-test-slave-mmhs"
server_key: "c3f7ba4f62cbf47050cd12bda9d09233"
server_key_iv: "f1035ad4607cb544192f1cd054df55fa"
server_key_version: "encryptionkey@0"
I20260924 00:56:48.657403 16285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:48.694392 16285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:48.698953 16285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:48.699239 16285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:48.702170 16285 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:56:48.702286 16285 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:48.702464 16285 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:56:48.702548 16285 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 00:56:48 dist-test-slave-mmhs krb5kdc[16029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211408, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.64.130@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM
I20260924 00:56:48.730636 16169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:48.703079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:34359 (local address 127.8.64.190:36845)
0924 00:56:48.703291 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:48.703299 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:48.704949 (+ 1650us) server_negotiation.cc:407] Connection header received
0924 00:56:48.708326 (+ 3377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:48.708335 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:48.708459 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:48.708606 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:48.710150 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:48.711577 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:48.714878 (+ 3301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:48.715761 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:48.715941 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:48.721979 (+ 6038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:48.721986 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:48.721992 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:48.722037 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:48.725958 (+ 3921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:48.727266 (+ 1308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:48.727274 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:48.727278 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:48.727455 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:48.728143 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:48.728150 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:48.728156 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:48.728459 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:48.728582 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:48.729891 (+ 1309us) server_negotiation.cc:299] Negotiation successful
0924 00:56:48.730159 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260924 00:56:48.732965 16309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:48.703766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:36845 (local address 127.8.64.130:34359)
0924 00:56:48.704738 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:48.704804 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:48.707904 (+ 3100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:48.708874 (+ 970us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:48.708882 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:48.709401 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:48.709964 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:48.709981 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:48.711807 (+ 1826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:48.711815 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:48.714713 (+ 2898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:48.714728 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:48.716066 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:48.716076 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:48.716449 (+ 373us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:48.717349 (+ 900us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:48.717394 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:48.721680 (+ 4286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:48.726147 (+ 4467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:48.726161 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:48.726184 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:48.727110 (+ 926us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:48.727597 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:48.727604 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:48.727609 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:48.727985 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:48.728609 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:48.728625 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:48.729318 (+ 693us) client_negotiation.cc:769] Sending connection context
0924 00:56:48.729764 (+ 446us) client_negotiation.cc:240] Negotiation successful
0924 00:56:48.730595 (+ 831us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"thread_start_us":431,"threads_started":1}
I20260924 00:56:48.745370 16419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.130:45455 every 8 connection(s)
I20260924 00:56:48.745369 16285 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.130:45455
I20260924 00:56:48.747228 16285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
I20260924 00:56:48.747814 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 16285
I20260924 00:56:48.748032 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance
I20260924 00:56:48.748730 8450 external_mini_cluster.cc:1428] Setting key e9dd906548e1de5a7ae7389783fab819
I20260924 00:56:48.751848 16420 heartbeater.cc:347] Connected to a master server at 127.8.64.190:36845
I20260924 00:56:48.752140 16420 heartbeater.cc:464] Registering TS with master...
I20260924 00:56:48.753052 16420 heartbeater.cc:511] Master 127.8.64.190:36845 requested a full tablet report, sending...
I20260924 00:56:48.754585 16074 ts_manager.cc:194] Registered new tserver with Master: 456e100a017b491d8d11fd3a68031fd6 (127.8.64.130:45455)
I20260924 00:56:48.755820 16074 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.64.130@KRBTEST.COM'} at 127.8.64.130:34359
I20260924 00:56:48.757234 16169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:48.740944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:57507 (local address 127.8.64.190:36845)
0924 00:56:48.741147 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:48.741154 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:48.742134 (+ 980us) server_negotiation.cc:407] Connection header received
0924 00:56:48.742357 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:48.742362 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:48.742445 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:48.742558 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:48.743952 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:48.745064 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:48.747206 (+ 2142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:48.747699 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:48.747852 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:48.750134 (+ 2282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:48.750141 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:48.750147 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:48.750189 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:48.753959 (+ 3770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:48.754802 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:48.754814 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:48.754818 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:48.754975 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:48.755400 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:48.755405 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:48.755409 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:48.755980 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:48.756140 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:48.756638 (+ 498us) server_negotiation.cc:299] Negotiation successful
0924 00:56:48.756864 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":58}
I20260924 00:56:48.757478 16411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:48.741067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:36845 (local address 127.8.64.130:57507)
0924 00:56:48.742024 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:48.742040 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:48.742214 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:48.742761 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:48.742768 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:48.743435 (+ 667us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:48.743802 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:48.743812 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:48.745256 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:48.745262 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:48.747050 (+ 1788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:48.747067 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:48.747896 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:48.747902 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:48.748075 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:48.748739 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:48.748770 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:48.749891 (+ 1121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:48.754097 (+ 4206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:48.754105 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:48.754111 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:48.754688 (+ 577us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:48.755101 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:48.755109 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:48.755115 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:48.755273 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:48.756121 (+ 848us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:48.756129 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:48.756343 (+ 214us) client_negotiation.cc:769] Sending connection context
0924 00:56:48.756512 (+ 169us) client_negotiation.cc:240] Negotiation successful
0924 00:56:48.756878 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":832,"thread_start_us":697,"threads_started":1}
WARNING: no policy specified for oryx/127.8.64.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.64.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.64.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.131.keytab.
Entry for principal oryx/127.8.64.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.131.keytab.
I20260924 00:56:48.802392 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.131:0
--local_ip_for_outbound_sockets=127.8.64.131
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:36845
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.131.keytab
--principal=oryx/127.8.64.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:39031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:56:49.129148 16424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:49.129494 16424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:49.129711 16424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:49.137053 16424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:49.137187 16424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:49.137351 16424 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 00:56:49.137478 16424 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.131
I20260924 00:56:49.147132 16424 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:39031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.131.keytab
--principal=oryx/127.8.64.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.64.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.64.190:36845
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.131
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:49.148411 16424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:49.149724 16424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:49.162848 16429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:49.162916 16430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:49.163465 16432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:49.164103 16424 server_base.cc:1034] running on GCE node
I20260924 00:56:49.164839 16424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:49.165825 16424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:49.167065 16424 hybrid_clock.cc:648] HybridClock initialized: now 1790211409166989 us; error 58 us; skew 500 ppm
I20260924 00:56:49.167363 16424 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:49 dist-test-slave-mmhs krb5kdc[16029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211409, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.64.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:49.170478 16424 init.cc:377] Logged in from keytab as oryx/127.8.64.131@KRBTEST.COM (short username oryx)
I20260924 00:56:49.171830 16424 webserver.cc:460] Webserver started at http://127.8.64.131:38203/ using document root <none> and password file <none>
I20260924 00:56:49.172472 16424 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:49.172554 16424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:49.172806 16424 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:56:49.175794 16424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-2/data/instance:
uuid: "bed6c0acef1b447580fffb7275147fb8"
format_stamp: "Formatted at 2026-09-24 00:56:49 on dist-test-slave-mmhs"
server_key: "23bf52f337a8361bc759a42472eb7add"
server_key_iv: "1d20a3d29ccd853ad45611e439dca45e"
server_key_version: "encryptionkey@0"
I20260924 00:56:49.176641 16424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance:
uuid: "bed6c0acef1b447580fffb7275147fb8"
format_stamp: "Formatted at 2026-09-24 00:56:49 on dist-test-slave-mmhs"
server_key: "23bf52f337a8361bc759a42472eb7add"
server_key_iv: "1d20a3d29ccd853ad45611e439dca45e"
server_key_version: "encryptionkey@0"
I20260924 00:56:49.181775 16424 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260924 00:56:49.185494 16439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:49.187098 16424 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 00:56:49.187284 16424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-2/wal
uuid: "bed6c0acef1b447580fffb7275147fb8"
format_stamp: "Formatted at 2026-09-24 00:56:49 on dist-test-slave-mmhs"
server_key: "23bf52f337a8361bc759a42472eb7add"
server_key_iv: "1d20a3d29ccd853ad45611e439dca45e"
server_key_version: "encryptionkey@0"
I20260924 00:56:49.187518 16424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:49.216295 16281 heartbeater.cc:503] Master 127.8.64.190:36845 was elected leader, sending a full tablet report...
I20260924 00:56:49.234935 16424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:49.239027 16424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:49.239302 16424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:49.241874 16424 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:56:49.241953 16424 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:49.242144 16424 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:56:49.242233 16424 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 00:56:49 dist-test-slave-mmhs krb5kdc[16029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211409, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.64.131@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM
I20260924 00:56:49.265270 16169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:49.242884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:58851 (local address 127.8.64.190:36845)
0924 00:56:49.243076 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:49.243082 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:49.244577 (+ 1495us) server_negotiation.cc:407] Connection header received
0924 00:56:49.247813 (+ 3236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:49.247821 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:49.247933 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:49.248069 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:49.250429 (+ 2360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:49.251347 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:49.253631 (+ 2284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:49.254464 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:49.254613 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:49.258348 (+ 3735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:49.258354 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:49.258360 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:49.258407 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:49.261980 (+ 3573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:49.262869 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:49.262875 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:49.262879 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:49.263016 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:49.263459 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:49.263469 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:49.263472 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:49.263725 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:49.263875 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:49.264688 (+ 813us) server_negotiation.cc:299] Negotiation successful
0924 00:56:49.264904 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260924 00:56:49.266554 16448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:49.243463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:36845 (local address 127.8.64.131:58851)
0924 00:56:49.244382 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:49.244440 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:49.247433 (+ 2993us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:49.248364 (+ 931us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:49.248382 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:49.249226 (+ 844us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:49.250124 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:49.250154 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:49.251525 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:49.251533 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:49.253458 (+ 1925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:49.253478 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:49.254726 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:49.254732 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:49.254910 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:49.255599 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:49.255620 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:49.258093 (+ 2473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:49.262139 (+ 4046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:49.262148 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:49.262165 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:49.262735 (+ 570us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:49.263139 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:49.263144 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:49.263148 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:49.263329 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:49.263862 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:49.263870 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:49.264228 (+ 358us) client_negotiation.cc:769] Sending connection context
0924 00:56:49.264556 (+ 328us) client_negotiation.cc:240] Negotiation successful
0924 00:56:49.265084 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":424,"threads_started":1}
I20260924 00:56:49.288571 16169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:49.273097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:47613 (local address 127.8.64.190:36845)
0924 00:56:49.273308 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:49.273315 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:49.273849 (+ 534us) server_negotiation.cc:407] Connection header received
0924 00:56:49.274052 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:49.274057 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:49.274130 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:49.274240 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:49.275764 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:49.276677 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:49.279225 (+ 2548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:49.279782 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:49.279941 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:49.282611 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:49.282620 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:49.282628 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:49.282671 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:49.285743 (+ 3072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:49.286650 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:49.286655 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:49.286658 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:49.286789 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:49.287236 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:49.287239 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:49.287241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:49.287507 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:49.287612 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:49.288165 (+ 553us) server_negotiation.cc:299] Negotiation successful
0924 00:56:49.288314 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260924 00:56:49.289156 16509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:49.272794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:36845 (local address 127.8.64.131:47613)
0924 00:56:49.273747 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:49.273762 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:49.273924 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:49.274331 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:49.274339 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:49.275108 (+ 769us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:49.275582 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:49.275598 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:49.276817 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:49.276836 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:49.279059 (+ 2223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:49.279082 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:49.280011 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:49.280024 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:49.280271 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:49.280979 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:49.281009 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:49.282381 (+ 1372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:49.285871 (+ 3490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:49.285876 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:49.285881 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:49.286506 (+ 625us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:49.286902 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:49.286910 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:49.286916 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:49.287118 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:49.287623 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:49.287631 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:49.287896 (+ 265us) client_negotiation.cc:769] Sending connection context
0924 00:56:49.288078 (+ 182us) client_negotiation.cc:240] Negotiation successful
0924 00:56:49.288502 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":836,"thread_start_us":690,"threads_started":1}
I20260924 00:56:49.299659 16424 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.131:36111
I20260924 00:56:49.299687 16558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.131:36111 every 8 connection(s)
I20260924 00:56:49.300858 16424 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
I20260924 00:56:49.303020 16559 heartbeater.cc:347] Connected to a master server at 127.8.64.190:36845
I20260924 00:56:49.303247 16559 heartbeater.cc:464] Registering TS with master...
I20260924 00:56:49.303875 16559 heartbeater.cc:511] Master 127.8.64.190:36845 requested a full tablet report, sending...
I20260924 00:56:49.305135 16074 ts_manager.cc:194] Registered new tserver with Master: bed6c0acef1b447580fffb7275147fb8 (127.8.64.131:36111)
I20260924 00:56:49.305359 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 16424
I20260924 00:56:49.305532 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance
I20260924 00:56:49.305960 8450 external_mini_cluster.cc:1428] Setting key 099578d91d821c31ed738e0e58c150f7
I20260924 00:56:49.306064 16074 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.64.131@KRBTEST.COM'} at 127.8.64.131:58851
I20260924 00:56:49.308933 8450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 24 00:56:49 dist-test-slave-mmhs krb5kdc[16029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211409, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 24 00:56:49 dist-test-slave-mmhs krb5kdc[16029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:56:49 dist-test-slave-mmhs krb5kdc[16029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
W20260924 00:56:49.358956 16169 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:56:49.348219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34592 (local address 127.8.64.190:36845)
0924 00:56:49.348419 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:49.348424 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:49.348736 (+ 312us) server_negotiation.cc:407] Connection header received
0924 00:56:49.348873 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:49.348877 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:49.348944 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:49.349048 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:49.350458 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:49.351263 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:49.352545 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:49.352988 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:49.353091 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:49.358752 (+ 5661us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34592: BlockingRecv error: recv got EOF from 127.0.0.1:34592 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Sep 24 00:56:49 dist-test-slave-mmhs krb5kdc[16029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211409, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM
I20260924 00:56:49.375260 16169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:49.362187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34602 (local address 127.8.64.190:36845)
0924 00:56:49.362439 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:49.362447 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:49.362737 (+ 290us) server_negotiation.cc:407] Connection header received
0924 00:56:49.362951 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:49.362955 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:49.363054 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:49.363157 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:49.364193 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:49.364913 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:49.366139 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:49.366607 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:49.366734 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:49.370483 (+ 3749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:49.370487 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:49.370491 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:49.370521 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:49.372801 (+ 2280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:49.373413 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:49.373425 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:49.373427 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:49.373533 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:49.373907 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:49.373913 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:49.373916 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:49.374167 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:49.374474 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:49.374734 (+ 260us) server_negotiation.cc:299] Negotiation successful
0924 00:56:49.374950 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260924 00:56:49.378975 16074 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34602:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:56:49.385757 16074 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:56:49.419085 16580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:49.398576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60358 (local address 127.8.64.129:35697)
0924 00:56:49.410367 (+ 11791us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:49.410378 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:49.410996 (+ 618us) server_negotiation.cc:407] Connection header received
0924 00:56:49.411115 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:49.411124 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:49.411390 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:49.411589 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:56:49.412911 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:49.414572 (+ 1661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:49.416689 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:49.417907 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:49.418043 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:49.418245 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:49.418603 (+ 358us) server_negotiation.cc:299] Negotiation successful
0924 00:56:49.418765 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11608,"thread_start_us":11493,"threads_started":1}
I20260924 00:56:49.419438 16577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:49.397960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:35697 (local address 127.0.0.1:60358)
0924 00:56:49.410814 (+ 12854us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:49.410832 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:49.410950 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:49.411915 (+ 965us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:49.411926 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:49.411967 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:56:49.412575 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:49.412593 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:49.414724 (+ 2131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:49.414729 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:49.416534 (+ 1805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:49.416546 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:49.418111 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:49.418118 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:49.418355 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:49.418410 (+ 55us) client_negotiation.cc:769] Sending connection context
0924 00:56:49.418551 (+ 141us) client_negotiation.cc:240] Negotiation successful
0924 00:56:49.418737 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12726,"mutex_wait_us":85,"thread_start_us":12288,"threads_started":1}
I20260924 00:56:49.420042 16579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:49.397850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.130:45455 (local address 127.0.0.1:53460)
0924 00:56:49.409735 (+ 11885us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:49.409800 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:49.410007 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:49.412985 (+ 2978us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:49.412989 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:49.413014 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:56:49.413382 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:49.413394 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:49.415109 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:49.415115 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:49.417417 (+ 2302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:49.417428 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:49.419207 (+ 1779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:49.419215 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:49.419368 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:49.419400 (+ 32us) client_negotiation.cc:769] Sending connection context
0924 00:56:49.419505 (+ 105us) client_negotiation.cc:240] Negotiation successful
0924 00:56:49.419640 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11536,"mutex_wait_us":39,"thread_start_us":12693,"threads_started":1}
I20260924 00:56:49.420730 16578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:49.397990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53460 (local address 127.8.64.130:45455)
0924 00:56:49.412163 (+ 14173us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:49.412174 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:49.412218 (+ 44us) server_negotiation.cc:407] Connection header received
0924 00:56:49.412319 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:49.412326 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:49.412565 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:49.412749 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:56:49.413561 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:49.414968 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:49.417569 (+ 2601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:49.418760 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:49.418894 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:49.419072 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:49.419579 (+ 507us) server_negotiation.cc:299] Negotiation successful
0924 00:56:49.419732 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14005,"thread_start_us":13895,"threads_started":1}
I20260924 00:56:49.421775 16582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:49.411460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55990 (local address 127.8.64.131:36111)
0924 00:56:49.413248 (+ 1788us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:49.413258 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:49.413293 (+ 35us) server_negotiation.cc:407] Connection header received
0924 00:56:49.413385 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:49.413392 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:49.413632 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:49.414570 (+ 938us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:56:49.414917 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:49.416566 (+ 1649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:49.419055 (+ 2489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:49.420445 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:49.420571 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:49.420790 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:49.421289 (+ 499us) server_negotiation.cc:299] Negotiation successful
0924 00:56:49.421441 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1609,"thread_start_us":1427,"threads_started":1}
I20260924 00:56:49.422374 16581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:49.411231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.131:36111 (local address 127.0.0.1:55990)
0924 00:56:49.412508 (+ 1277us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:49.412529 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:49.412637 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:49.413868 (+ 1231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:49.413873 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:49.413899 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:56:49.414329 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:49.414342 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:49.416744 (+ 2402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:49.416749 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:49.418861 (+ 2112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:49.418874 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:49.420931 (+ 2057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:49.420937 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:49.421095 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:49.421125 (+ 30us) client_negotiation.cc:769] Sending connection context
0924 00:56:49.421222 (+ 97us) client_negotiation.cc:240] Negotiation successful
0924 00:56:49.421353 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1156,"mutex_wait_us":61,"thread_start_us":2254,"threads_started":1}
I20260924 00:56:49.423656 16211 tablet_service.cc:1467] Processing CreateTablet for tablet 0c9d136feaaf41eaab36128a47f2645c (DEFAULT_TABLE table=test-table [id=98faed2c71614e90b92c45c2b963141c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:56:49.424113 16350 tablet_service.cc:1467] Processing CreateTablet for tablet 0c9d136feaaf41eaab36128a47f2645c (DEFAULT_TABLE table=test-table [id=98faed2c71614e90b92c45c2b963141c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:56:49.425143 16211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c9d136feaaf41eaab36128a47f2645c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:49.425361 16350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c9d136feaaf41eaab36128a47f2645c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:49.425050 16489 tablet_service.cc:1467] Processing CreateTablet for tablet 0c9d136feaaf41eaab36128a47f2645c (DEFAULT_TABLE table=test-table [id=98faed2c71614e90b92c45c2b963141c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:56:49.426182 16489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c9d136feaaf41eaab36128a47f2645c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:49.438498 16583 tablet_bootstrap.cc:492] T 0c9d136feaaf41eaab36128a47f2645c P bed6c0acef1b447580fffb7275147fb8: Bootstrap starting.
I20260924 00:56:49.441653 16583 tablet_bootstrap.cc:654] T 0c9d136feaaf41eaab36128a47f2645c P bed6c0acef1b447580fffb7275147fb8: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:49.442987 16583 log.cc:824] T 0c9d136feaaf41eaab36128a47f2645c P bed6c0acef1b447580fffb7275147fb8: Log is configured to *not* fsync() on all Append() calls
I20260924 00:56:49.443578 16585 tablet_bootstrap.cc:492] T 0c9d136feaaf41eaab36128a47f2645c P adfab66f8ae74a76ac0d1d09df6a617c: Bootstrap starting.
I20260924 00:56:49.443979 16584 tablet_bootstrap.cc:492] T 0c9d136feaaf41eaab36128a47f2645c P 456e100a017b491d8d11fd3a68031fd6: Bootstrap starting.
I20260924 00:56:49.446758 16583 tablet_bootstrap.cc:492] T 0c9d136feaaf41eaab36128a47f2645c P bed6c0acef1b447580fffb7275147fb8: No bootstrap required, opened a new log
I20260924 00:56:49.447058 16583 ts_tablet_manager.cc:1397] T 0c9d136feaaf41eaab36128a47f2645c P bed6c0acef1b447580fffb7275147fb8: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260924 00:56:49.447316 16584 tablet_bootstrap.cc:654] T 0c9d136feaaf41eaab36128a47f2645c P 456e100a017b491d8d11fd3a68031fd6: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:49.447737 16585 tablet_bootstrap.cc:654] T 0c9d136feaaf41eaab36128a47f2645c P adfab66f8ae74a76ac0d1d09df6a617c: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:49.448421 16584 log.cc:824] T 0c9d136feaaf41eaab36128a47f2645c P 456e100a017b491d8d11fd3a68031fd6: Log is configured to *not* fsync() on all Append() calls
I20260924 00:56:49.448900 16585 log.cc:824] T 0c9d136feaaf41eaab36128a47f2645c P adfab66f8ae74a76ac0d1d09df6a617c: Log is configured to *not* fsync() on all Append() calls
I20260924 00:56:49.451354 16585 tablet_bootstrap.cc:492] T 0c9d136feaaf41eaab36128a47f2645c P adfab66f8ae74a76ac0d1d09df6a617c: No bootstrap required, opened a new log
I20260924 00:56:49.451450 16584 tablet_bootstrap.cc:492] T 0c9d136feaaf41eaab36128a47f2645c P 456e100a017b491d8d11fd3a68031fd6: No bootstrap required, opened a new log
I20260924 00:56:49.451650 16585 ts_tablet_manager.cc:1397] T 0c9d136feaaf41eaab36128a47f2645c P adfab66f8ae74a76ac0d1d09df6a617c: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260924 00:56:49.451709 16584 ts_tablet_manager.cc:1397] T 0c9d136feaaf41eaab36128a47f2645c P 456e100a017b491d8d11fd3a68031fd6: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.004s
I20260924 00:56:49.457765 16583 raft_consensus.cc:348] T 0c9d136feaaf41eaab36128a47f2645c P bed6c0acef1b447580fffb7275147fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } }
I20260924 00:56:49.458202 16583 raft_consensus.cc:374] T 0c9d136feaaf41eaab36128a47f2645c P bed6c0acef1b447580fffb7275147fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:49.458320 16583 raft_consensus.cc:729] T 0c9d136feaaf41eaab36128a47f2645c P bed6c0acef1b447580fffb7275147fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bed6c0acef1b447580fffb7275147fb8, State: Initialized, Role: FOLLOWER
I20260924 00:56:49.458479 16585 raft_consensus.cc:348] T 0c9d136feaaf41eaab36128a47f2645c P adfab66f8ae74a76ac0d1d09df6a617c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } }
I20260924 00:56:49.458928 16585 raft_consensus.cc:374] T 0c9d136feaaf41eaab36128a47f2645c P adfab66f8ae74a76ac0d1d09df6a617c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:49.459030 16585 raft_consensus.cc:729] T 0c9d136feaaf41eaab36128a47f2645c P adfab66f8ae74a76ac0d1d09df6a617c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adfab66f8ae74a76ac0d1d09df6a617c, State: Initialized, Role: FOLLOWER
I20260924 00:56:49.458971 16583 consensus_queue.cc:260] T 0c9d136feaaf41eaab36128a47f2645c P bed6c0acef1b447580fffb7275147fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } }
I20260924 00:56:49.459620 16585 consensus_queue.cc:260] T 0c9d136feaaf41eaab36128a47f2645c P adfab66f8ae74a76ac0d1d09df6a617c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } }
I20260924 00:56:49.463317 16584 raft_consensus.cc:348] T 0c9d136feaaf41eaab36128a47f2645c P 456e100a017b491d8d11fd3a68031fd6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } }
I20260924 00:56:49.463835 16584 raft_consensus.cc:374] T 0c9d136feaaf41eaab36128a47f2645c P 456e100a017b491d8d11fd3a68031fd6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:49.463951 16584 raft_consensus.cc:729] T 0c9d136feaaf41eaab36128a47f2645c P 456e100a017b491d8d11fd3a68031fd6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 456e100a017b491d8d11fd3a68031fd6, State: Initialized, Role: FOLLOWER
I20260924 00:56:49.464651 16584 consensus_queue.cc:260] T 0c9d136feaaf41eaab36128a47f2645c P 456e100a017b491d8d11fd3a68031fd6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } }
I20260924 00:56:49.466019 16420 heartbeater.cc:503] Master 127.8.64.190:36845 was elected leader, sending a full tablet report...
I20260924 00:56:49.466281 16559 heartbeater.cc:503] Master 127.8.64.190:36845 was elected leader, sending a full tablet report...
I20260924 00:56:49.467284 16583 ts_tablet_manager.cc:1428] T 0c9d136feaaf41eaab36128a47f2645c P bed6c0acef1b447580fffb7275147fb8: Time spent starting tablet: real 0.020s user 0.015s sys 0.004s
I20260924 00:56:49.468739 16584 ts_tablet_manager.cc:1428] T 0c9d136feaaf41eaab36128a47f2645c P 456e100a017b491d8d11fd3a68031fd6: Time spent starting tablet: real 0.017s user 0.007s sys 0.009s
I20260924 00:56:49.470148 16585 ts_tablet_manager.cc:1428] T 0c9d136feaaf41eaab36128a47f2645c P adfab66f8ae74a76ac0d1d09df6a617c: Time spent starting tablet: real 0.018s user 0.015s sys 0.004s
W20260924 00:56:49.501523 16421 tablet.cc:2369] T 0c9d136feaaf41eaab36128a47f2645c P 456e100a017b491d8d11fd3a68031fd6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 00:56:49.552491 16560 tablet.cc:2369] T 0c9d136feaaf41eaab36128a47f2645c P bed6c0acef1b447580fffb7275147fb8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:56:49.635300 16591 raft_consensus.cc:482] T 0c9d136feaaf41eaab36128a47f2645c P 456e100a017b491d8d11fd3a68031fd6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:56:49.635667 16591 raft_consensus.cc:504] T 0c9d136feaaf41eaab36128a47f2645c P 456e100a017b491d8d11fd3a68031fd6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } }
I20260924 00:56:49.637382 16591 leader_election.cc:288] T 0c9d136feaaf41eaab36128a47f2645c P 456e100a017b491d8d11fd3a68031fd6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers adfab66f8ae74a76ac0d1d09df6a617c (127.8.64.129:35697), bed6c0acef1b447580fffb7275147fb8 (127.8.64.131:36111)
I20260924 00:56:49.644734 16580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:49.637823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:34583 (local address 127.8.64.129:35697)
0924 00:56:49.638025 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:49.638030 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:49.638680 (+ 650us) server_negotiation.cc:407] Connection header received
0924 00:56:49.638954 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:49.638963 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:49.639060 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:49.639123 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:56:49.639907 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:49.640794 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:49.642719 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:49.643668 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:49.643842 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:49.643927 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:49.644368 (+ 441us) server_negotiation.cc:299] Negotiation successful
0924 00:56:49.644489 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260924 00:56:49.644822 16592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:49.637785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:35697 (local address 127.8.64.130:34583)
0924 00:56:49.638559 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:49.638581 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:49.638722 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:49.639328 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:49.639332 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:49.639369 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:56:49.639758 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:49.639771 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:49.640931 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:49.640936 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:49.642568 (+ 1632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:49.642580 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:49.643963 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:49.643971 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:49.644173 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:49.644209 (+ 36us) client_negotiation.cc:769] Sending connection context
0924 00:56:49.644338 (+ 129us) client_negotiation.cc:240] Negotiation successful
0924 00:56:49.644465 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"thread_start_us":385,"threads_started":1}
I20260924 00:56:49.645305 16593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:49.638699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.131:36111 (local address 127.8.64.130:37683)
0924 00:56:49.639267 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:49.639281 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:49.639418 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:49.639763 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:49.639765 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:49.639784 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:56:49.640188 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:49.640200 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:49.641281 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:49.641285 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:49.642810 (+ 1525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:49.642823 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:49.644351 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:49.644362 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:49.644636 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:49.644660 (+ 24us) client_negotiation.cc:769] Sending connection context
0924 00:56:49.644769 (+ 109us) client_negotiation.cc:240] Negotiation successful
0924 00:56:49.644914 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"spinlock_wait_cycles":17152,"thread_start_us":293,"threads_started":1}
I20260924 00:56:49.645655 16582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:49.638989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:37683 (local address 127.8.64.131:36111)
0924 00:56:49.639305 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:49.639311 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:49.639343 (+ 32us) server_negotiation.cc:407] Connection header received
0924 00:56:49.639562 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:49.639570 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:49.639643 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:49.639743 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:56:49.640344 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:49.641142 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:49.642954 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:49.643980 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:49.644143 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:49.644211 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:49.644796 (+ 585us) server_negotiation.cc:299] Negotiation successful
0924 00:56:49.645243 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171}
I20260924 00:56:49.645881 16234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c9d136feaaf41eaab36128a47f2645c" candidate_uuid: "456e100a017b491d8d11fd3a68031fd6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" is_pre_election: true
I20260924 00:56:49.646494 16234 raft_consensus.cc:2454] T 0c9d136feaaf41eaab36128a47f2645c P adfab66f8ae74a76ac0d1d09df6a617c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 456e100a017b491d8d11fd3a68031fd6 in term 0.
I20260924 00:56:49.646566 16514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c9d136feaaf41eaab36128a47f2645c" candidate_uuid: "456e100a017b491d8d11fd3a68031fd6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bed6c0acef1b447580fffb7275147fb8" is_pre_election: true
I20260924 00:56:49.647152 16514 raft_consensus.cc:2454] T 0c9d136feaaf41eaab36128a47f2645c P bed6c0acef1b447580fffb7275147fb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 456e100a017b491d8d11fd3a68031fd6 in term 0.
I20260924 00:56:49.647240 16301 leader_election.cc:302] T 0c9d136feaaf41eaab36128a47f2645c P 456e100a017b491d8d11fd3a68031fd6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 456e100a017b491d8d11fd3a68031fd6, adfab66f8ae74a76ac0d1d09df6a617c; no voters:
I20260924 00:56:49.647673 16591 raft_consensus.cc:2781] T 0c9d136feaaf41eaab36128a47f2645c P 456e100a017b491d8d11fd3a68031fd6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:56:49.647835 16591 raft_consensus.cc:482] T 0c9d136feaaf41eaab36128a47f2645c P 456e100a017b491d8d11fd3a68031fd6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:56:49.647967 16591 raft_consensus.cc:3037] T 0c9d136feaaf41eaab36128a47f2645c P 456e100a017b491d8d11fd3a68031fd6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:49.650094 16591 raft_consensus.cc:504] T 0c9d136feaaf41eaab36128a47f2645c P 456e100a017b491d8d11fd3a68031fd6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } }
I20260924 00:56:49.650683 16591 leader_election.cc:288] T 0c9d136feaaf41eaab36128a47f2645c P 456e100a017b491d8d11fd3a68031fd6 [CANDIDATE]: Term 1 election: Requested vote from peers adfab66f8ae74a76ac0d1d09df6a617c (127.8.64.129:35697), bed6c0acef1b447580fffb7275147fb8 (127.8.64.131:36111)
I20260924 00:56:49.651189 16234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c9d136feaaf41eaab36128a47f2645c" candidate_uuid: "456e100a017b491d8d11fd3a68031fd6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adfab66f8ae74a76ac0d1d09df6a617c"
I20260924 00:56:49.651244 16514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c9d136feaaf41eaab36128a47f2645c" candidate_uuid: "456e100a017b491d8d11fd3a68031fd6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bed6c0acef1b447580fffb7275147fb8"
I20260924 00:56:49.651419 16514 raft_consensus.cc:3037] T 0c9d136feaaf41eaab36128a47f2645c P bed6c0acef1b447580fffb7275147fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:49.651419 16234 raft_consensus.cc:3037] T 0c9d136feaaf41eaab36128a47f2645c P adfab66f8ae74a76ac0d1d09df6a617c [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:49.654587 16234 raft_consensus.cc:2454] T 0c9d136feaaf41eaab36128a47f2645c P adfab66f8ae74a76ac0d1d09df6a617c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 456e100a017b491d8d11fd3a68031fd6 in term 1.
I20260924 00:56:49.654587 16514 raft_consensus.cc:2454] T 0c9d136feaaf41eaab36128a47f2645c P bed6c0acef1b447580fffb7275147fb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 456e100a017b491d8d11fd3a68031fd6 in term 1.
I20260924 00:56:49.654978 16301 leader_election.cc:302] T 0c9d136feaaf41eaab36128a47f2645c P 456e100a017b491d8d11fd3a68031fd6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 456e100a017b491d8d11fd3a68031fd6, bed6c0acef1b447580fffb7275147fb8; no voters:
I20260924 00:56:49.655242 16591 raft_consensus.cc:2781] T 0c9d136feaaf41eaab36128a47f2645c P 456e100a017b491d8d11fd3a68031fd6 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:56:49.655682 16591 raft_consensus.cc:686] T 0c9d136feaaf41eaab36128a47f2645c P 456e100a017b491d8d11fd3a68031fd6 [term 1 LEADER]: Becoming Leader. State: Replica: 456e100a017b491d8d11fd3a68031fd6, State: Running, Role: LEADER
I20260924 00:56:49.656423 16591 consensus_queue.cc:237] T 0c9d136feaaf41eaab36128a47f2645c P 456e100a017b491d8d11fd3a68031fd6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } }
I20260924 00:56:49.660951 16074 catalog_manager.cc:5495] T 0c9d136feaaf41eaab36128a47f2645c P 456e100a017b491d8d11fd3a68031fd6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 456e100a017b491d8d11fd3a68031fd6 (127.8.64.130). New cstate: current_term: 1 leader_uuid: "456e100a017b491d8d11fd3a68031fd6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } health_report { overall_health: UNKNOWN } } }
I20260924 00:56:49.702747 16578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:49.697341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53468 (local address 127.8.64.130:45455)
0924 00:56:49.697554 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:49.697559 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:49.697575 (+ 16us) server_negotiation.cc:407] Connection header received
0924 00:56:49.697660 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:49.697663 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:49.697735 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:49.697864 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:56:49.698175 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:49.698944 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:49.700248 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:49.700842 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:49.700964 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:49.701432 (+ 468us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:56:49.702028 (+ 596us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:56:49.702144 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:49.702355 (+ 211us) server_negotiation.cc:299] Negotiation successful
0924 00:56:49.702429 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
W20260924 00:56:49.708438 16282 tablet.cc:2369] T 0c9d136feaaf41eaab36128a47f2645c P adfab66f8ae74a76ac0d1d09df6a617c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:56:49.710908 16234 raft_consensus.cc:1261] T 0c9d136feaaf41eaab36128a47f2645c P adfab66f8ae74a76ac0d1d09df6a617c [term 1 FOLLOWER]: Refusing update from remote peer 456e100a017b491d8d11fd3a68031fd6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:56:49.710908 16514 raft_consensus.cc:1261] T 0c9d136feaaf41eaab36128a47f2645c P bed6c0acef1b447580fffb7275147fb8 [term 1 FOLLOWER]: Refusing update from remote peer 456e100a017b491d8d11fd3a68031fd6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:56:49.711824 16594 consensus_queue.cc:1035] T 0c9d136feaaf41eaab36128a47f2645c P 456e100a017b491d8d11fd3a68031fd6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:56:49.712174 16591 consensus_queue.cc:1035] T 0c9d136feaaf41eaab36128a47f2645c P 456e100a017b491d8d11fd3a68031fd6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:56:49.736402 16073 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34602:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:56:49.736655 16073 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34602:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:56:49.740219 16073 catalog_manager.cc:5782] T 00000000000000000000000000000000 P debc27d99cb84de7940c1a40cfe8fff2: Sending DeleteTablet for 3 replicas of tablet 0c9d136feaaf41eaab36128a47f2645c
I20260924 00:56:49.741470 16489 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c9d136feaaf41eaab36128a47f2645c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:56:49 UTC) from {username='oryx', principal='oryx/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:55990
I20260924 00:56:49.741433 16350 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c9d136feaaf41eaab36128a47f2645c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:56:49 UTC) from {username='oryx', principal='oryx/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:53460
I20260924 00:56:49.742472 16211 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c9d136feaaf41eaab36128a47f2645c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:56:49 UTC) from {username='oryx', principal='oryx/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:60358
I20260924 00:56:49.742609 16073 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34602:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:56:49.743309 16073 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:56:49.744951 16609 tablet_replica.cc:330] T 0c9d136feaaf41eaab36128a47f2645c P adfab66f8ae74a76ac0d1d09df6a617c: stopping tablet replica
I20260924 00:56:49.745368 16609 raft_consensus.cc:2229] T 0c9d136feaaf41eaab36128a47f2645c P adfab66f8ae74a76ac0d1d09df6a617c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:56:49.745849 16609 raft_consensus.cc:2258] T 0c9d136feaaf41eaab36128a47f2645c P adfab66f8ae74a76ac0d1d09df6a617c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:56:49.756989 16607 tablet_replica.cc:330] T 0c9d136feaaf41eaab36128a47f2645c P 456e100a017b491d8d11fd3a68031fd6: stopping tablet replica
I20260924 00:56:49.757328 16349 tablet_service.cc:1467] Processing CreateTablet for tablet 2cd4dd4c87874e289cf947c1d7f9ded4 (DEFAULT_TABLE table=test-table [id=83b77b7fdcc14ac39886d86c798e1a1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:56:49.757462 16607 raft_consensus.cc:2229] T 0c9d136feaaf41eaab36128a47f2645c P 456e100a017b491d8d11fd3a68031fd6 [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:56:49.757477 16488 tablet_service.cc:1467] Processing CreateTablet for tablet 2cd4dd4c87874e289cf947c1d7f9ded4 (DEFAULT_TABLE table=test-table [id=83b77b7fdcc14ac39886d86c798e1a1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:56:49.757989 16349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cd4dd4c87874e289cf947c1d7f9ded4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:49.757997 16488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cd4dd4c87874e289cf947c1d7f9ded4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:49.758169 16607 raft_consensus.cc:2258] T 0c9d136feaaf41eaab36128a47f2645c P 456e100a017b491d8d11fd3a68031fd6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:56:49.759142 16608 tablet_replica.cc:330] T 0c9d136feaaf41eaab36128a47f2645c P bed6c0acef1b447580fffb7275147fb8: stopping tablet replica
I20260924 00:56:49.759549 16608 raft_consensus.cc:2229] T 0c9d136feaaf41eaab36128a47f2645c P bed6c0acef1b447580fffb7275147fb8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:56:49.758692 16211 tablet_service.cc:1467] Processing CreateTablet for tablet 2cd4dd4c87874e289cf947c1d7f9ded4 (DEFAULT_TABLE table=test-table [id=83b77b7fdcc14ac39886d86c798e1a1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:56:49.760149 16608 raft_consensus.cc:2258] T 0c9d136feaaf41eaab36128a47f2645c P bed6c0acef1b447580fffb7275147fb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:56:49.760396 16211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cd4dd4c87874e289cf947c1d7f9ded4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:49.764715 16583 tablet_bootstrap.cc:492] T 2cd4dd4c87874e289cf947c1d7f9ded4 P bed6c0acef1b447580fffb7275147fb8: Bootstrap starting.
I20260924 00:56:49.766424 16583 tablet_bootstrap.cc:654] T 2cd4dd4c87874e289cf947c1d7f9ded4 P bed6c0acef1b447580fffb7275147fb8: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:49.768018 16585 tablet_bootstrap.cc:492] T 2cd4dd4c87874e289cf947c1d7f9ded4 P adfab66f8ae74a76ac0d1d09df6a617c: Bootstrap starting.
I20260924 00:56:49.768126 16584 tablet_bootstrap.cc:492] T 2cd4dd4c87874e289cf947c1d7f9ded4 P 456e100a017b491d8d11fd3a68031fd6: Bootstrap starting.
I20260924 00:56:49.769709 16585 tablet_bootstrap.cc:654] T 2cd4dd4c87874e289cf947c1d7f9ded4 P adfab66f8ae74a76ac0d1d09df6a617c: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:49.769721 16584 tablet_bootstrap.cc:654] T 2cd4dd4c87874e289cf947c1d7f9ded4 P 456e100a017b491d8d11fd3a68031fd6: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:49.774777 16608 ts_tablet_manager.cc:1905] T 0c9d136feaaf41eaab36128a47f2645c P bed6c0acef1b447580fffb7275147fb8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:56:49.778502 16583 tablet_bootstrap.cc:492] T 2cd4dd4c87874e289cf947c1d7f9ded4 P bed6c0acef1b447580fffb7275147fb8: No bootstrap required, opened a new log
I20260924 00:56:49.782477 16583 ts_tablet_manager.cc:1397] T 2cd4dd4c87874e289cf947c1d7f9ded4 P bed6c0acef1b447580fffb7275147fb8: Time spent bootstrapping tablet: real 0.018s user 0.013s sys 0.000s
I20260924 00:56:49.783313 16583 raft_consensus.cc:348] T 2cd4dd4c87874e289cf947c1d7f9ded4 P bed6c0acef1b447580fffb7275147fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } }
I20260924 00:56:49.783576 16583 raft_consensus.cc:374] T 2cd4dd4c87874e289cf947c1d7f9ded4 P bed6c0acef1b447580fffb7275147fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:49.783671 16583 raft_consensus.cc:729] T 2cd4dd4c87874e289cf947c1d7f9ded4 P bed6c0acef1b447580fffb7275147fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bed6c0acef1b447580fffb7275147fb8, State: Initialized, Role: FOLLOWER
I20260924 00:56:49.783938 16583 consensus_queue.cc:260] T 2cd4dd4c87874e289cf947c1d7f9ded4 P bed6c0acef1b447580fffb7275147fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } }
I20260924 00:56:49.784515 16583 ts_tablet_manager.cc:1428] T 2cd4dd4c87874e289cf947c1d7f9ded4 P bed6c0acef1b447580fffb7275147fb8: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260924 00:56:49.787431 16609 ts_tablet_manager.cc:1905] T 0c9d136feaaf41eaab36128a47f2645c P adfab66f8ae74a76ac0d1d09df6a617c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:56:49.792773 16607 ts_tablet_manager.cc:1905] T 0c9d136feaaf41eaab36128a47f2645c P 456e100a017b491d8d11fd3a68031fd6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:56:49.793272 16584 tablet_bootstrap.cc:492] T 2cd4dd4c87874e289cf947c1d7f9ded4 P 456e100a017b491d8d11fd3a68031fd6: No bootstrap required, opened a new log
I20260924 00:56:49.793475 16584 ts_tablet_manager.cc:1397] T 2cd4dd4c87874e289cf947c1d7f9ded4 P 456e100a017b491d8d11fd3a68031fd6: Time spent bootstrapping tablet: real 0.025s user 0.003s sys 0.006s
I20260924 00:56:49.793919 16585 tablet_bootstrap.cc:492] T 2cd4dd4c87874e289cf947c1d7f9ded4 P adfab66f8ae74a76ac0d1d09df6a617c: No bootstrap required, opened a new log
I20260924 00:56:49.794139 16585 ts_tablet_manager.cc:1397] T 2cd4dd4c87874e289cf947c1d7f9ded4 P adfab66f8ae74a76ac0d1d09df6a617c: Time spent bootstrapping tablet: real 0.026s user 0.005s sys 0.011s
I20260924 00:56:49.794687 16584 raft_consensus.cc:348] T 2cd4dd4c87874e289cf947c1d7f9ded4 P 456e100a017b491d8d11fd3a68031fd6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } }
I20260924 00:56:49.794986 16584 raft_consensus.cc:374] T 2cd4dd4c87874e289cf947c1d7f9ded4 P 456e100a017b491d8d11fd3a68031fd6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:49.795136 16584 raft_consensus.cc:729] T 2cd4dd4c87874e289cf947c1d7f9ded4 P 456e100a017b491d8d11fd3a68031fd6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 456e100a017b491d8d11fd3a68031fd6, State: Initialized, Role: FOLLOWER
I20260924 00:56:49.795102 16585 raft_consensus.cc:348] T 2cd4dd4c87874e289cf947c1d7f9ded4 P adfab66f8ae74a76ac0d1d09df6a617c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } }
I20260924 00:56:49.795329 16585 raft_consensus.cc:374] T 2cd4dd4c87874e289cf947c1d7f9ded4 P adfab66f8ae74a76ac0d1d09df6a617c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:49.795430 16585 raft_consensus.cc:729] T 2cd4dd4c87874e289cf947c1d7f9ded4 P adfab66f8ae74a76ac0d1d09df6a617c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adfab66f8ae74a76ac0d1d09df6a617c, State: Initialized, Role: FOLLOWER
I20260924 00:56:49.795715 16585 consensus_queue.cc:260] T 2cd4dd4c87874e289cf947c1d7f9ded4 P adfab66f8ae74a76ac0d1d09df6a617c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } }
I20260924 00:56:49.795709 16584 consensus_queue.cc:260] T 2cd4dd4c87874e289cf947c1d7f9ded4 P 456e100a017b491d8d11fd3a68031fd6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } }
I20260924 00:56:49.797648 16608 ts_tablet_manager.cc:1918] T 0c9d136feaaf41eaab36128a47f2645c P bed6c0acef1b447580fffb7275147fb8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:56:49.797820 16608 log.cc:1196] T 0c9d136feaaf41eaab36128a47f2645c P bed6c0acef1b447580fffb7275147fb8: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-2/wal/wals/0c9d136feaaf41eaab36128a47f2645c
I20260924 00:56:49.798375 16608 ts_tablet_manager.cc:1939] T 0c9d136feaaf41eaab36128a47f2645c P bed6c0acef1b447580fffb7275147fb8: Deleting consensus metadata
I20260924 00:56:49.799170 16609 ts_tablet_manager.cc:1918] T 0c9d136feaaf41eaab36128a47f2645c P adfab66f8ae74a76ac0d1d09df6a617c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:56:49.799327 16609 log.cc:1196] T 0c9d136feaaf41eaab36128a47f2645c P adfab66f8ae74a76ac0d1d09df6a617c: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-0/wal/wals/0c9d136feaaf41eaab36128a47f2645c
I20260924 00:56:49.799351 16584 ts_tablet_manager.cc:1428] T 2cd4dd4c87874e289cf947c1d7f9ded4 P 456e100a017b491d8d11fd3a68031fd6: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260924 00:56:49.799837 16609 ts_tablet_manager.cc:1939] T 0c9d136feaaf41eaab36128a47f2645c P adfab66f8ae74a76ac0d1d09df6a617c: Deleting consensus metadata
I20260924 00:56:49.800295 16058 catalog_manager.cc:4841] TS bed6c0acef1b447580fffb7275147fb8 (127.8.64.131:36111): tablet 0c9d136feaaf41eaab36128a47f2645c (table test-table [id=98faed2c71614e90b92c45c2b963141c]) successfully deleted
I20260924 00:56:49.800985 16585 ts_tablet_manager.cc:1428] T 2cd4dd4c87874e289cf947c1d7f9ded4 P adfab66f8ae74a76ac0d1d09df6a617c: Time spent starting tablet: real 0.007s user 0.000s sys 0.002s
I20260924 00:56:49.804983 16058 catalog_manager.cc:4841] TS adfab66f8ae74a76ac0d1d09df6a617c (127.8.64.129:35697): tablet 0c9d136feaaf41eaab36128a47f2645c (table test-table [id=98faed2c71614e90b92c45c2b963141c]) successfully deleted
I20260924 00:56:49.806545 16607 ts_tablet_manager.cc:1918] T 0c9d136feaaf41eaab36128a47f2645c P 456e100a017b491d8d11fd3a68031fd6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:56:49.806748 16607 log.cc:1196] T 0c9d136feaaf41eaab36128a47f2645c P 456e100a017b491d8d11fd3a68031fd6: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-1/wal/wals/0c9d136feaaf41eaab36128a47f2645c
I20260924 00:56:49.807248 16607 ts_tablet_manager.cc:1939] T 0c9d136feaaf41eaab36128a47f2645c P 456e100a017b491d8d11fd3a68031fd6: Deleting consensus metadata
I20260924 00:56:49.808827 16059 catalog_manager.cc:4841] TS 456e100a017b491d8d11fd3a68031fd6 (127.8.64.130:45455): tablet 0c9d136feaaf41eaab36128a47f2645c (table test-table [id=98faed2c71614e90b92c45c2b963141c]) successfully deleted
I20260924 00:56:49.959988 16594 raft_consensus.cc:482] T 2cd4dd4c87874e289cf947c1d7f9ded4 P 456e100a017b491d8d11fd3a68031fd6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:56:49.960273 16594 raft_consensus.cc:504] T 2cd4dd4c87874e289cf947c1d7f9ded4 P 456e100a017b491d8d11fd3a68031fd6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } }
I20260924 00:56:49.961215 16594 leader_election.cc:288] T 2cd4dd4c87874e289cf947c1d7f9ded4 P 456e100a017b491d8d11fd3a68031fd6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers adfab66f8ae74a76ac0d1d09df6a617c (127.8.64.129:35697), bed6c0acef1b447580fffb7275147fb8 (127.8.64.131:36111)
I20260924 00:56:49.961778 16234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cd4dd4c87874e289cf947c1d7f9ded4" candidate_uuid: "456e100a017b491d8d11fd3a68031fd6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" is_pre_election: true
I20260924 00:56:49.961838 16514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cd4dd4c87874e289cf947c1d7f9ded4" candidate_uuid: "456e100a017b491d8d11fd3a68031fd6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bed6c0acef1b447580fffb7275147fb8" is_pre_election: true
I20260924 00:56:49.962038 16234 raft_consensus.cc:2454] T 2cd4dd4c87874e289cf947c1d7f9ded4 P adfab66f8ae74a76ac0d1d09df6a617c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 456e100a017b491d8d11fd3a68031fd6 in term 0.
I20260924 00:56:49.962070 16514 raft_consensus.cc:2454] T 2cd4dd4c87874e289cf947c1d7f9ded4 P bed6c0acef1b447580fffb7275147fb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 456e100a017b491d8d11fd3a68031fd6 in term 0.
I20260924 00:56:49.962502 16301 leader_election.cc:302] T 2cd4dd4c87874e289cf947c1d7f9ded4 P 456e100a017b491d8d11fd3a68031fd6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 456e100a017b491d8d11fd3a68031fd6, bed6c0acef1b447580fffb7275147fb8; no voters:
I20260924 00:56:49.962791 16594 raft_consensus.cc:2781] T 2cd4dd4c87874e289cf947c1d7f9ded4 P 456e100a017b491d8d11fd3a68031fd6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:56:49.962917 16594 raft_consensus.cc:482] T 2cd4dd4c87874e289cf947c1d7f9ded4 P 456e100a017b491d8d11fd3a68031fd6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:56:49.963032 16594 raft_consensus.cc:3037] T 2cd4dd4c87874e289cf947c1d7f9ded4 P 456e100a017b491d8d11fd3a68031fd6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:49.966364 16594 raft_consensus.cc:504] T 2cd4dd4c87874e289cf947c1d7f9ded4 P 456e100a017b491d8d11fd3a68031fd6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } }
I20260924 00:56:49.967099 16594 leader_election.cc:288] T 2cd4dd4c87874e289cf947c1d7f9ded4 P 456e100a017b491d8d11fd3a68031fd6 [CANDIDATE]: Term 1 election: Requested vote from peers adfab66f8ae74a76ac0d1d09df6a617c (127.8.64.129:35697), bed6c0acef1b447580fffb7275147fb8 (127.8.64.131:36111)
I20260924 00:56:49.967485 16234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cd4dd4c87874e289cf947c1d7f9ded4" candidate_uuid: "456e100a017b491d8d11fd3a68031fd6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adfab66f8ae74a76ac0d1d09df6a617c"
I20260924 00:56:49.967712 16234 raft_consensus.cc:3037] T 2cd4dd4c87874e289cf947c1d7f9ded4 P adfab66f8ae74a76ac0d1d09df6a617c [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:49.967665 16514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cd4dd4c87874e289cf947c1d7f9ded4" candidate_uuid: "456e100a017b491d8d11fd3a68031fd6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bed6c0acef1b447580fffb7275147fb8"
I20260924 00:56:49.967856 16514 raft_consensus.cc:3037] T 2cd4dd4c87874e289cf947c1d7f9ded4 P bed6c0acef1b447580fffb7275147fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:49.969849 16234 raft_consensus.cc:2454] T 2cd4dd4c87874e289cf947c1d7f9ded4 P adfab66f8ae74a76ac0d1d09df6a617c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 456e100a017b491d8d11fd3a68031fd6 in term 1.
I20260924 00:56:49.970237 16301 leader_election.cc:302] T 2cd4dd4c87874e289cf947c1d7f9ded4 P 456e100a017b491d8d11fd3a68031fd6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 456e100a017b491d8d11fd3a68031fd6, adfab66f8ae74a76ac0d1d09df6a617c; no voters:
I20260924 00:56:49.970544 16594 raft_consensus.cc:2781] T 2cd4dd4c87874e289cf947c1d7f9ded4 P 456e100a017b491d8d11fd3a68031fd6 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:56:49.970745 16594 raft_consensus.cc:686] T 2cd4dd4c87874e289cf947c1d7f9ded4 P 456e100a017b491d8d11fd3a68031fd6 [term 1 LEADER]: Becoming Leader. State: Replica: 456e100a017b491d8d11fd3a68031fd6, State: Running, Role: LEADER
I20260924 00:56:49.970801 16514 raft_consensus.cc:2454] T 2cd4dd4c87874e289cf947c1d7f9ded4 P bed6c0acef1b447580fffb7275147fb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 456e100a017b491d8d11fd3a68031fd6 in term 1.
I20260924 00:56:49.971143 16594 consensus_queue.cc:237] T 2cd4dd4c87874e289cf947c1d7f9ded4 P 456e100a017b491d8d11fd3a68031fd6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } }
I20260924 00:56:49.973778 16073 catalog_manager.cc:5495] T 2cd4dd4c87874e289cf947c1d7f9ded4 P 456e100a017b491d8d11fd3a68031fd6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 456e100a017b491d8d11fd3a68031fd6 (127.8.64.130). New cstate: current_term: 1 leader_uuid: "456e100a017b491d8d11fd3a68031fd6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } health_report { overall_health: UNKNOWN } } }
Sep 24 00:56:50 dist-test-slave-mmhs krb5kdc[16029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211407, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.64.190@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM
I20260924 00:56:50.044744 16620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:50.024716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34614 (local address 127.8.64.190:36845)
0924 00:56:50.025581 (+ 865us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:50.025591 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:50.025608 (+ 17us) server_negotiation.cc:407] Connection header received
0924 00:56:50.025646 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:50.025649 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:50.025710 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:50.025824 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:50.026902 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:50.027670 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:50.028914 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:50.029319 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:50.029472 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:50.038244 (+ 8772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:50.038253 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:50.038258 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:50.038319 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:50.041702 (+ 3383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:50.042647 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:50.042651 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:50.042653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:50.042767 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:50.043238 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:50.043241 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:50.043243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:50.043505 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:50.043687 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:50.044147 (+ 460us) server_negotiation.cc:299] Negotiation successful
0924 00:56:50.044480 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":158,"server-negotiator.queue_time_us":732,"thread_start_us":565,"threads_started":1}
I20260924 00:56:50.044829 16619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:50.024569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:36845 (local address 127.0.0.1:34614)
0924 00:56:50.025247 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:50.025273 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:50.025459 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:50.025850 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:50.025854 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:50.026581 (+ 727us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:50.026760 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:50.026768 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:50.027835 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:50.027842 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:50.028775 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:50.028785 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:50.029513 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:50.029520 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:50.029678 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:50.035739 (+ 6061us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:50.035771 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:50.038012 (+ 2241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:50.041895 (+ 3883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:50.041907 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:50.041916 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:50.042524 (+ 608us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:50.042917 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:50.042922 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:50.042926 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:50.043115 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:50.043646 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:50.043655 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:50.043862 (+ 207us) client_negotiation.cc:769] Sending connection context
0924 00:56:50.044048 (+ 186us) client_negotiation.cc:240] Negotiation successful
0924 00:56:50.044364 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":369,"threads_started":1}
I20260924 00:56:50.050203 16073 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:34614:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260924 00:56:50.051270 16073 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:56:50.057934 16211 tablet_service.cc:1467] Processing CreateTablet for tablet ef402deffb6b447196b26dc780ce0846 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=75f6415699624d9c93b3bb70644a8e8c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 00:56:50.058350 16349 tablet_service.cc:1467] Processing CreateTablet for tablet ef402deffb6b447196b26dc780ce0846 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=75f6415699624d9c93b3bb70644a8e8c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 00:56:50.058346 16488 tablet_service.cc:1467] Processing CreateTablet for tablet ef402deffb6b447196b26dc780ce0846 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=75f6415699624d9c93b3bb70644a8e8c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 00:56:50.058586 16211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef402deffb6b447196b26dc780ce0846. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:50.058843 16488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef402deffb6b447196b26dc780ce0846. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:50.058849 16349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef402deffb6b447196b26dc780ce0846. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:50.066154 16584 tablet_bootstrap.cc:492] T ef402deffb6b447196b26dc780ce0846 P 456e100a017b491d8d11fd3a68031fd6: Bootstrap starting.
I20260924 00:56:50.066172 16583 tablet_bootstrap.cc:492] T ef402deffb6b447196b26dc780ce0846 P bed6c0acef1b447580fffb7275147fb8: Bootstrap starting.
I20260924 00:56:50.067540 16584 tablet_bootstrap.cc:654] T ef402deffb6b447196b26dc780ce0846 P 456e100a017b491d8d11fd3a68031fd6: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:50.067866 16583 tablet_bootstrap.cc:654] T ef402deffb6b447196b26dc780ce0846 P bed6c0acef1b447580fffb7275147fb8: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:50.070259 16584 tablet_bootstrap.cc:492] T ef402deffb6b447196b26dc780ce0846 P 456e100a017b491d8d11fd3a68031fd6: No bootstrap required, opened a new log
I20260924 00:56:50.070470 16584 ts_tablet_manager.cc:1397] T ef402deffb6b447196b26dc780ce0846 P 456e100a017b491d8d11fd3a68031fd6: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260924 00:56:50.070771 16583 tablet_bootstrap.cc:492] T ef402deffb6b447196b26dc780ce0846 P bed6c0acef1b447580fffb7275147fb8: No bootstrap required, opened a new log
I20260924 00:56:50.070933 16583 ts_tablet_manager.cc:1397] T ef402deffb6b447196b26dc780ce0846 P bed6c0acef1b447580fffb7275147fb8: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260924 00:56:50.071269 16584 raft_consensus.cc:348] T ef402deffb6b447196b26dc780ce0846 P 456e100a017b491d8d11fd3a68031fd6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } }
I20260924 00:56:50.071512 16584 raft_consensus.cc:374] T ef402deffb6b447196b26dc780ce0846 P 456e100a017b491d8d11fd3a68031fd6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:50.071614 16584 raft_consensus.cc:729] T ef402deffb6b447196b26dc780ce0846 P 456e100a017b491d8d11fd3a68031fd6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 456e100a017b491d8d11fd3a68031fd6, State: Initialized, Role: FOLLOWER
I20260924 00:56:50.071640 16583 raft_consensus.cc:348] T ef402deffb6b447196b26dc780ce0846 P bed6c0acef1b447580fffb7275147fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } }
I20260924 00:56:50.071906 16583 raft_consensus.cc:374] T ef402deffb6b447196b26dc780ce0846 P bed6c0acef1b447580fffb7275147fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:50.071995 16583 raft_consensus.cc:729] T ef402deffb6b447196b26dc780ce0846 P bed6c0acef1b447580fffb7275147fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bed6c0acef1b447580fffb7275147fb8, State: Initialized, Role: FOLLOWER
I20260924 00:56:50.071920 16584 consensus_queue.cc:260] T ef402deffb6b447196b26dc780ce0846 P 456e100a017b491d8d11fd3a68031fd6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } }
I20260924 00:56:50.072229 16583 consensus_queue.cc:260] T ef402deffb6b447196b26dc780ce0846 P bed6c0acef1b447580fffb7275147fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } }
I20260924 00:56:50.072487 16584 ts_tablet_manager.cc:1428] T ef402deffb6b447196b26dc780ce0846 P 456e100a017b491d8d11fd3a68031fd6: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 00:56:50.072777 16594 tablet_replica.cc:439] T ef402deffb6b447196b26dc780ce0846 P 456e100a017b491d8d11fd3a68031fd6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } } }
I20260924 00:56:50.073053 16594 tablet_replica.cc:442] T ef402deffb6b447196b26dc780ce0846 P 456e100a017b491d8d11fd3a68031fd6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 00:56:50.073428 16589 tablet_replica.cc:439] T ef402deffb6b447196b26dc780ce0846 P bed6c0acef1b447580fffb7275147fb8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } } }
I20260924 00:56:50.073716 16589 tablet_replica.cc:442] T ef402deffb6b447196b26dc780ce0846 P bed6c0acef1b447580fffb7275147fb8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 00:56:50.074592 16583 ts_tablet_manager.cc:1428] T ef402deffb6b447196b26dc780ce0846 P bed6c0acef1b447580fffb7275147fb8: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260924 00:56:50.075737 16585 tablet_bootstrap.cc:492] T ef402deffb6b447196b26dc780ce0846 P adfab66f8ae74a76ac0d1d09df6a617c: Bootstrap starting.
I20260924 00:56:50.077464 16585 tablet_bootstrap.cc:654] T ef402deffb6b447196b26dc780ce0846 P adfab66f8ae74a76ac0d1d09df6a617c: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:50.080255 16585 tablet_bootstrap.cc:492] T ef402deffb6b447196b26dc780ce0846 P adfab66f8ae74a76ac0d1d09df6a617c: No bootstrap required, opened a new log
I20260924 00:56:50.080426 16585 ts_tablet_manager.cc:1397] T ef402deffb6b447196b26dc780ce0846 P adfab66f8ae74a76ac0d1d09df6a617c: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.002s
I20260924 00:56:50.080987 16585 raft_consensus.cc:348] T ef402deffb6b447196b26dc780ce0846 P adfab66f8ae74a76ac0d1d09df6a617c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } }
I20260924 00:56:50.081193 16585 raft_consensus.cc:374] T ef402deffb6b447196b26dc780ce0846 P adfab66f8ae74a76ac0d1d09df6a617c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:50.081296 16585 raft_consensus.cc:729] T ef402deffb6b447196b26dc780ce0846 P adfab66f8ae74a76ac0d1d09df6a617c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adfab66f8ae74a76ac0d1d09df6a617c, State: Initialized, Role: FOLLOWER
I20260924 00:56:50.081533 16585 consensus_queue.cc:260] T ef402deffb6b447196b26dc780ce0846 P adfab66f8ae74a76ac0d1d09df6a617c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } }
I20260924 00:56:50.082008 16590 tablet_replica.cc:439] T ef402deffb6b447196b26dc780ce0846 P adfab66f8ae74a76ac0d1d09df6a617c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } } }
I20260924 00:56:50.082398 16590 tablet_replica.cc:442] T ef402deffb6b447196b26dc780ce0846 P adfab66f8ae74a76ac0d1d09df6a617c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 00:56:50.082856 16585 ts_tablet_manager.cc:1428] T ef402deffb6b447196b26dc780ce0846 P adfab66f8ae74a76ac0d1d09df6a617c: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260924 00:56:50.126904 16589 raft_consensus.cc:482] T ef402deffb6b447196b26dc780ce0846 P bed6c0acef1b447580fffb7275147fb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:56:50.127166 16589 raft_consensus.cc:504] T ef402deffb6b447196b26dc780ce0846 P bed6c0acef1b447580fffb7275147fb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } }
I20260924 00:56:50.128970 16589 leader_election.cc:288] T ef402deffb6b447196b26dc780ce0846 P bed6c0acef1b447580fffb7275147fb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers adfab66f8ae74a76ac0d1d09df6a617c (127.8.64.129:35697), 456e100a017b491d8d11fd3a68031fd6 (127.8.64.130:45455)
I20260924 00:56:50.136912 16624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:50.129157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:35697 (local address 127.8.64.131:40691)
0924 00:56:50.129920 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:50.129947 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:50.130150 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:50.130606 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:50.130612 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:50.130636 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:56:50.130907 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:50.130920 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:50.132122 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:50.132130 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:50.134400 (+ 2270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:50.134415 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:50.136140 (+ 1725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:50.136146 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:50.136316 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:50.136362 (+ 46us) client_negotiation.cc:769] Sending connection context
0924 00:56:50.136459 (+ 97us) client_negotiation.cc:240] Negotiation successful
0924 00:56:50.136587 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":420,"threads_started":1}
I20260924 00:56:50.136941 16580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:50.129396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:40691 (local address 127.8.64.129:35697)
0924 00:56:50.129717 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:50.129723 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:50.130040 (+ 317us) server_negotiation.cc:407] Connection header received
0924 00:56:50.130335 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:50.130341 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:50.130473 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:50.130573 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:56:50.131090 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:50.131997 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:50.134566 (+ 2569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:50.135808 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:50.135979 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:50.136050 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:50.136511 (+ 461us) server_negotiation.cc:299] Negotiation successful
0924 00:56:50.136647 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197}
I20260924 00:56:50.137261 16578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:50.129379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:58229 (local address 127.8.64.130:45455)
0924 00:56:50.129564 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:50.129570 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:50.130400 (+ 830us) server_negotiation.cc:407] Connection header received
0924 00:56:50.130586 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:50.130591 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:50.130656 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:50.130743 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:56:50.131358 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:50.132485 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:50.134761 (+ 2276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:50.136045 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:50.136188 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:50.136257 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:50.136817 (+ 560us) server_negotiation.cc:299] Negotiation successful
0924 00:56:50.136969 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260924 00:56:50.137879 16625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:50.129157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.130:45455 (local address 127.8.64.131:58229)
0924 00:56:50.130248 (+ 1091us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:50.130267 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:50.130446 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:50.130760 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:50.130768 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:50.130793 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:56:50.131175 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:50.131188 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:50.132604 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:50.132609 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:50.134618 (+ 2009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:50.134627 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:50.136421 (+ 1794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:50.136450 (+ 29us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:50.136635 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:50.136666 (+ 31us) client_negotiation.cc:769] Sending connection context
0924 00:56:50.137443 (+ 777us) client_negotiation.cc:240] Negotiation successful
0924 00:56:50.137569 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":960,"mutex_wait_us":178,"thread_start_us":597,"threads_started":1}
I20260924 00:56:50.137840 16234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef402deffb6b447196b26dc780ce0846" candidate_uuid: "bed6c0acef1b447580fffb7275147fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" is_pre_election: true
I20260924 00:56:50.138123 16234 raft_consensus.cc:2454] T ef402deffb6b447196b26dc780ce0846 P adfab66f8ae74a76ac0d1d09df6a617c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bed6c0acef1b447580fffb7275147fb8 in term 0.
I20260924 00:56:50.138628 16440 leader_election.cc:302] T ef402deffb6b447196b26dc780ce0846 P bed6c0acef1b447580fffb7275147fb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: adfab66f8ae74a76ac0d1d09df6a617c, bed6c0acef1b447580fffb7275147fb8; no voters:
I20260924 00:56:50.138705 16370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef402deffb6b447196b26dc780ce0846" candidate_uuid: "bed6c0acef1b447580fffb7275147fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "456e100a017b491d8d11fd3a68031fd6" is_pre_election: true
I20260924 00:56:50.139036 16589 raft_consensus.cc:2781] T ef402deffb6b447196b26dc780ce0846 P bed6c0acef1b447580fffb7275147fb8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:56:50.139151 16589 raft_consensus.cc:482] T ef402deffb6b447196b26dc780ce0846 P bed6c0acef1b447580fffb7275147fb8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:56:50.139236 16589 raft_consensus.cc:3037] T ef402deffb6b447196b26dc780ce0846 P bed6c0acef1b447580fffb7275147fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:50.139175 16370 raft_consensus.cc:2454] T ef402deffb6b447196b26dc780ce0846 P 456e100a017b491d8d11fd3a68031fd6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bed6c0acef1b447580fffb7275147fb8 in term 0.
I20260924 00:56:50.142213 16589 raft_consensus.cc:504] T ef402deffb6b447196b26dc780ce0846 P bed6c0acef1b447580fffb7275147fb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } }
I20260924 00:56:50.142932 16589 leader_election.cc:288] T ef402deffb6b447196b26dc780ce0846 P bed6c0acef1b447580fffb7275147fb8 [CANDIDATE]: Term 1 election: Requested vote from peers adfab66f8ae74a76ac0d1d09df6a617c (127.8.64.129:35697), 456e100a017b491d8d11fd3a68031fd6 (127.8.64.130:45455)
I20260924 00:56:50.143369 16370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef402deffb6b447196b26dc780ce0846" candidate_uuid: "bed6c0acef1b447580fffb7275147fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "456e100a017b491d8d11fd3a68031fd6"
I20260924 00:56:50.143380 16234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef402deffb6b447196b26dc780ce0846" candidate_uuid: "bed6c0acef1b447580fffb7275147fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adfab66f8ae74a76ac0d1d09df6a617c"
I20260924 00:56:50.143585 16234 raft_consensus.cc:3037] T ef402deffb6b447196b26dc780ce0846 P adfab66f8ae74a76ac0d1d09df6a617c [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:50.143581 16370 raft_consensus.cc:3037] T ef402deffb6b447196b26dc780ce0846 P 456e100a017b491d8d11fd3a68031fd6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:50.146183 16370 raft_consensus.cc:2454] T ef402deffb6b447196b26dc780ce0846 P 456e100a017b491d8d11fd3a68031fd6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bed6c0acef1b447580fffb7275147fb8 in term 1.
I20260924 00:56:50.146406 16234 raft_consensus.cc:2454] T ef402deffb6b447196b26dc780ce0846 P adfab66f8ae74a76ac0d1d09df6a617c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bed6c0acef1b447580fffb7275147fb8 in term 1.
I20260924 00:56:50.146576 16441 leader_election.cc:302] T ef402deffb6b447196b26dc780ce0846 P bed6c0acef1b447580fffb7275147fb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 456e100a017b491d8d11fd3a68031fd6, bed6c0acef1b447580fffb7275147fb8; no voters:
I20260924 00:56:50.146880 16589 raft_consensus.cc:2781] T ef402deffb6b447196b26dc780ce0846 P bed6c0acef1b447580fffb7275147fb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:56:50.147321 16589 raft_consensus.cc:686] T ef402deffb6b447196b26dc780ce0846 P bed6c0acef1b447580fffb7275147fb8 [term 1 LEADER]: Becoming Leader. State: Replica: bed6c0acef1b447580fffb7275147fb8, State: Running, Role: LEADER
I20260924 00:56:50.147707 16589 consensus_queue.cc:237] T ef402deffb6b447196b26dc780ce0846 P bed6c0acef1b447580fffb7275147fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } }
I20260924 00:56:50.149544 16626 tablet_replica.cc:439] T ef402deffb6b447196b26dc780ce0846 P bed6c0acef1b447580fffb7275147fb8: TxnStatusTablet state changed. Reason: New leader bed6c0acef1b447580fffb7275147fb8. Latest consensus state: current_term: 1 leader_uuid: "bed6c0acef1b447580fffb7275147fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } } }
I20260924 00:56:50.149852 16626 tablet_replica.cc:442] T ef402deffb6b447196b26dc780ce0846 P bed6c0acef1b447580fffb7275147fb8: This TxnStatusTablet replica's current role is: LEADER
I20260924 00:56:50.150569 16628 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260924 00:56:50.150784 16628 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260924 00:56:50.151504 16073 catalog_manager.cc:5495] T ef402deffb6b447196b26dc780ce0846 P bed6c0acef1b447580fffb7275147fb8 reported cstate change: term changed from 0 to 1, leader changed from <none> to bed6c0acef1b447580fffb7275147fb8 (127.8.64.131). New cstate: current_term: 1 leader_uuid: "bed6c0acef1b447580fffb7275147fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } health_report { overall_health: HEALTHY } } }
I20260924 00:56:50.173820 16619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:50.163916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.131:36111 (local address 127.0.0.1:55992)
0924 00:56:50.164270 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:50.164293 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:50.164443 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:50.167224 (+ 2781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:50.167229 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:50.167244 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 00:56:50.167434 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:50.167444 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:50.169115 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:50.169121 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:50.170836 (+ 1715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:50.170850 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:50.171836 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:50.171842 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:50.172034 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:50.172100 (+ 66us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 00:56:50.173229 (+ 1129us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 00:56:50.173250 (+ 21us) client_negotiation.cc:769] Sending connection context
0924 00:56:50.173344 (+ 94us) client_negotiation.cc:240] Negotiation successful
0924 00:56:50.173507 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198}
I20260924 00:56:50.173871 16631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:50.163996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55992 (local address 127.8.64.131:36111)
0924 00:56:50.166793 (+ 2797us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:50.166799 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:50.166823 (+ 24us) server_negotiation.cc:407] Connection header received
0924 00:56:50.166911 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:50.166918 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:50.167015 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:50.167176 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:56:50.167622 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:50.168973 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:50.170987 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:50.171677 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:50.171816 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:50.172331 (+ 515us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:56:50.173067 (+ 736us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:56:50.173279 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:50.173373 (+ 94us) server_negotiation.cc:299] Negotiation successful
0924 00:56:50.173472 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2645,"thread_start_us":2501,"threads_started":1}
I20260924 00:56:50.178771 16234 raft_consensus.cc:1261] T ef402deffb6b447196b26dc780ce0846 P adfab66f8ae74a76ac0d1d09df6a617c [term 1 FOLLOWER]: Refusing update from remote peer bed6c0acef1b447580fffb7275147fb8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:56:50.179028 16370 raft_consensus.cc:1261] T ef402deffb6b447196b26dc780ce0846 P 456e100a017b491d8d11fd3a68031fd6 [term 1 FOLLOWER]: Refusing update from remote peer bed6c0acef1b447580fffb7275147fb8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:56:50.179541 16626 consensus_queue.cc:1035] T ef402deffb6b447196b26dc780ce0846 P bed6c0acef1b447580fffb7275147fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:56:50.179858 16589 consensus_queue.cc:1035] T ef402deffb6b447196b26dc780ce0846 P bed6c0acef1b447580fffb7275147fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:56:50.192827 16590 tablet_replica.cc:439] T ef402deffb6b447196b26dc780ce0846 P adfab66f8ae74a76ac0d1d09df6a617c: TxnStatusTablet state changed. Reason: New leader bed6c0acef1b447580fffb7275147fb8. Latest consensus state: current_term: 1 leader_uuid: "bed6c0acef1b447580fffb7275147fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } } }
I20260924 00:56:50.193090 16590 tablet_replica.cc:442] T ef402deffb6b447196b26dc780ce0846 P adfab66f8ae74a76ac0d1d09df6a617c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 00:56:50.193439 16594 tablet_replica.cc:439] T ef402deffb6b447196b26dc780ce0846 P 456e100a017b491d8d11fd3a68031fd6: TxnStatusTablet state changed. Reason: New leader bed6c0acef1b447580fffb7275147fb8. Latest consensus state: current_term: 1 leader_uuid: "bed6c0acef1b447580fffb7275147fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } } }
I20260924 00:56:50.193702 16594 tablet_replica.cc:442] T ef402deffb6b447196b26dc780ce0846 P 456e100a017b491d8d11fd3a68031fd6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 00:56:50.195010 16626 tablet_replica.cc:439] T ef402deffb6b447196b26dc780ce0846 P bed6c0acef1b447580fffb7275147fb8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bed6c0acef1b447580fffb7275147fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } } }
I20260924 00:56:50.195281 16626 tablet_replica.cc:442] T ef402deffb6b447196b26dc780ce0846 P bed6c0acef1b447580fffb7275147fb8: This TxnStatusTablet replica's current role is: LEADER
I20260924 00:56:50.195462 16598 mvcc.cc:205] Tried to move back new op lower bound from 7332705936085004288 to 7332705935968419840. Current Snapshot: MvccSnapshot[applied={T|T < 7332705936085004288}]
I20260924 00:56:50.197861 16590 tablet_replica.cc:439] T ef402deffb6b447196b26dc780ce0846 P adfab66f8ae74a76ac0d1d09df6a617c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bed6c0acef1b447580fffb7275147fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } } }
I20260924 00:56:50.198098 16590 tablet_replica.cc:442] T ef402deffb6b447196b26dc780ce0846 P adfab66f8ae74a76ac0d1d09df6a617c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 00:56:50.208745 16626 tablet_replica.cc:439] T ef402deffb6b447196b26dc780ce0846 P bed6c0acef1b447580fffb7275147fb8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bed6c0acef1b447580fffb7275147fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } } }
I20260924 00:56:50.209030 16626 tablet_replica.cc:442] T ef402deffb6b447196b26dc780ce0846 P bed6c0acef1b447580fffb7275147fb8: This TxnStatusTablet replica's current role is: LEADER
I20260924 00:56:50.210233 16597 mvcc.cc:205] Tried to move back new op lower bound from 7332705936085004288 to 7332705935968419840. Current Snapshot: MvccSnapshot[applied={T|T < 7332705936085004288}]
I20260924 00:56:50.211001 16594 tablet_replica.cc:439] T ef402deffb6b447196b26dc780ce0846 P 456e100a017b491d8d11fd3a68031fd6: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bed6c0acef1b447580fffb7275147fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } } }
I20260924 00:56:50.211259 16594 tablet_replica.cc:442] T ef402deffb6b447196b26dc780ce0846 P 456e100a017b491d8d11fd3a68031fd6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 00:56:50.227259 16631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:50.220539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:52279 (local address 127.8.64.131:36111)
0924 00:56:50.220722 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:50.220730 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:50.221042 (+ 312us) server_negotiation.cc:407] Connection header received
0924 00:56:50.221234 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:50.221240 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:50.221321 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:50.221418 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:56:50.221965 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:50.223085 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:50.225104 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:50.225795 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:50.225916 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:50.226502 (+ 586us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:56:50.226630 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:56:50.226717 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:50.226876 (+ 159us) server_negotiation.cc:299] Negotiation successful
0924 00:56:50.226954 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260924 00:56:50.227371 16637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:50.220141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.131:36111 (local address 127.8.64.130:52279)
0924 00:56:50.220926 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:50.220945 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:50.221096 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:50.221463 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:50.221467 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:50.221481 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 00:56:50.221801 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:50.221814 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:50.223231 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:50.223237 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:50.224936 (+ 1699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:50.224953 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:50.225966 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:50.225974 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:50.226158 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:50.226234 (+ 76us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 00:56:50.226743 (+ 509us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 00:56:50.226754 (+ 11us) client_negotiation.cc:769] Sending connection context
0924 00:56:50.226846 (+ 92us) client_negotiation.cc:240] Negotiation successful
0924 00:56:50.226991 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":450,"threads_started":1}
I20260924 00:56:50.238339 16234 raft_consensus.cc:1261] T 2cd4dd4c87874e289cf947c1d7f9ded4 P adfab66f8ae74a76ac0d1d09df6a617c [term 1 FOLLOWER]: Refusing update from remote peer 456e100a017b491d8d11fd3a68031fd6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:56:50.238442 16514 raft_consensus.cc:1261] T 2cd4dd4c87874e289cf947c1d7f9ded4 P bed6c0acef1b447580fffb7275147fb8 [term 1 FOLLOWER]: Refusing update from remote peer 456e100a017b491d8d11fd3a68031fd6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:56:50.238957 16602 consensus_queue.cc:1035] T 2cd4dd4c87874e289cf947c1d7f9ded4 P 456e100a017b491d8d11fd3a68031fd6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:56:50.239464 16594 consensus_queue.cc:1035] T 2cd4dd4c87874e289cf947c1d7f9ded4 P 456e100a017b491d8d11fd3a68031fd6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:56:50.254943 16599 mvcc.cc:205] Tried to move back new op lower bound from 7332705936330661888 to 7332705935244075008. Current Snapshot: MvccSnapshot[applied={T|T < 7332705936330661888}]
I20260924 00:56:50.272945 16578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:50.266030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:45199 (local address 127.8.64.130:45455)
0924 00:56:50.266184 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:50.266192 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:50.267155 (+ 963us) server_negotiation.cc:407] Connection header received
0924 00:56:50.267301 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:50.267306 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:50.267387 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:50.267501 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:56:50.268005 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:50.269151 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:50.270984 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:50.271636 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:50.271763 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:50.272253 (+ 490us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:56:50.272411 (+ 158us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:56:50.272502 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:50.272658 (+ 156us) server_negotiation.cc:299] Negotiation successful
0924 00:56:50.272744 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260924 00:56:50.273207 16641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:50.266001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.130:45455 (local address 127.8.64.131:45199)
0924 00:56:50.267052 (+ 1051us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:50.267068 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:50.267177 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:50.267504 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:50.267507 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:50.267515 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 00:56:50.267855 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:50.267863 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:50.269275 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:50.269280 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:50.270854 (+ 1574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:50.270865 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:50.271802 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:50.271806 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:50.271989 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:50.272061 (+ 72us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 00:56:50.272521 (+ 460us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 00:56:50.272534 (+ 13us) client_negotiation.cc:769] Sending connection context
0924 00:56:50.272624 (+ 90us) client_negotiation.cc:240] Negotiation successful
0924 00:56:50.272759 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":856,"thread_start_us":703,"threads_started":1}
I20260924 00:56:50.309072 16074 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34602:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:56:50.309330 16074 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34602:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:56:50.312728 16074 catalog_manager.cc:5782] T 00000000000000000000000000000000 P debc27d99cb84de7940c1a40cfe8fff2: Sending DeleteTablet for 3 replicas of tablet 2cd4dd4c87874e289cf947c1d7f9ded4
I20260924 00:56:50.313747 16488 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cd4dd4c87874e289cf947c1d7f9ded4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:56:50 UTC) from {username='oryx', principal='oryx/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:55990
I20260924 00:56:50.313812 16211 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cd4dd4c87874e289cf947c1d7f9ded4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:56:50 UTC) from {username='oryx', principal='oryx/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:60358
I20260924 00:56:50.313819 16349 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cd4dd4c87874e289cf947c1d7f9ded4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:56:50 UTC) from {username='oryx', principal='oryx/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:53460
I20260924 00:56:50.344205 16645 tablet_replica.cc:330] T 2cd4dd4c87874e289cf947c1d7f9ded4 P bed6c0acef1b447580fffb7275147fb8: stopping tablet replica
I20260924 00:56:50.344810 16645 raft_consensus.cc:2229] T 2cd4dd4c87874e289cf947c1d7f9ded4 P bed6c0acef1b447580fffb7275147fb8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:56:50.345243 16645 raft_consensus.cc:2258] T 2cd4dd4c87874e289cf947c1d7f9ded4 P bed6c0acef1b447580fffb7275147fb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:56:50.348026 16645 ts_tablet_manager.cc:1905] T 2cd4dd4c87874e289cf947c1d7f9ded4 P bed6c0acef1b447580fffb7275147fb8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:56:50.348645 16646 tablet_replica.cc:330] T 2cd4dd4c87874e289cf947c1d7f9ded4 P 456e100a017b491d8d11fd3a68031fd6: stopping tablet replica
I20260924 00:56:50.348908 16646 raft_consensus.cc:2229] T 2cd4dd4c87874e289cf947c1d7f9ded4 P 456e100a017b491d8d11fd3a68031fd6 [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:56:50.349370 16646 raft_consensus.cc:2258] T 2cd4dd4c87874e289cf947c1d7f9ded4 P 456e100a017b491d8d11fd3a68031fd6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:56:50.350481 16646 ts_tablet_manager.cc:1905] T 2cd4dd4c87874e289cf947c1d7f9ded4 P 456e100a017b491d8d11fd3a68031fd6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:56:50.357549 16646 ts_tablet_manager.cc:1918] T 2cd4dd4c87874e289cf947c1d7f9ded4 P 456e100a017b491d8d11fd3a68031fd6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 00:56:50.357745 16646 log.cc:1196] T 2cd4dd4c87874e289cf947c1d7f9ded4 P 456e100a017b491d8d11fd3a68031fd6: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-1/wal/wals/2cd4dd4c87874e289cf947c1d7f9ded4
I20260924 00:56:50.358247 16646 ts_tablet_manager.cc:1939] T 2cd4dd4c87874e289cf947c1d7f9ded4 P 456e100a017b491d8d11fd3a68031fd6: Deleting consensus metadata
I20260924 00:56:50.359453 16059 catalog_manager.cc:4841] TS 456e100a017b491d8d11fd3a68031fd6 (127.8.64.130:45455): tablet 2cd4dd4c87874e289cf947c1d7f9ded4 (table test-table [id=83b77b7fdcc14ac39886d86c798e1a1c]) successfully deleted
I20260924 00:56:50.362689 16644 tablet_replica.cc:330] T 2cd4dd4c87874e289cf947c1d7f9ded4 P adfab66f8ae74a76ac0d1d09df6a617c: stopping tablet replica
I20260924 00:56:50.362888 16644 raft_consensus.cc:2229] T 2cd4dd4c87874e289cf947c1d7f9ded4 P adfab66f8ae74a76ac0d1d09df6a617c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:56:50.363132 16644 raft_consensus.cc:2258] T 2cd4dd4c87874e289cf947c1d7f9ded4 P adfab66f8ae74a76ac0d1d09df6a617c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:56:50.364246 16644 ts_tablet_manager.cc:1905] T 2cd4dd4c87874e289cf947c1d7f9ded4 P adfab66f8ae74a76ac0d1d09df6a617c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:56:50.371089 16645 ts_tablet_manager.cc:1918] T 2cd4dd4c87874e289cf947c1d7f9ded4 P bed6c0acef1b447580fffb7275147fb8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 00:56:50.371238 16645 log.cc:1196] T 2cd4dd4c87874e289cf947c1d7f9ded4 P bed6c0acef1b447580fffb7275147fb8: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-2/wal/wals/2cd4dd4c87874e289cf947c1d7f9ded4
I20260924 00:56:50.371714 16645 ts_tablet_manager.cc:1939] T 2cd4dd4c87874e289cf947c1d7f9ded4 P bed6c0acef1b447580fffb7275147fb8: Deleting consensus metadata
I20260924 00:56:50.372272 16644 ts_tablet_manager.cc:1918] T 2cd4dd4c87874e289cf947c1d7f9ded4 P adfab66f8ae74a76ac0d1d09df6a617c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 00:56:50.372462 16644 log.cc:1196] T 2cd4dd4c87874e289cf947c1d7f9ded4 P adfab66f8ae74a76ac0d1d09df6a617c: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-0/wal/wals/2cd4dd4c87874e289cf947c1d7f9ded4
I20260924 00:56:50.372972 16058 catalog_manager.cc:4841] TS bed6c0acef1b447580fffb7275147fb8 (127.8.64.131:36111): tablet 2cd4dd4c87874e289cf947c1d7f9ded4 (table test-table [id=83b77b7fdcc14ac39886d86c798e1a1c]) successfully deleted
I20260924 00:56:50.373621 16644 ts_tablet_manager.cc:1939] T 2cd4dd4c87874e289cf947c1d7f9ded4 P adfab66f8ae74a76ac0d1d09df6a617c: Deleting consensus metadata
I20260924 00:56:50.374769 16058 catalog_manager.cc:4841] TS adfab66f8ae74a76ac0d1d09df6a617c (127.8.64.129:35697): tablet 2cd4dd4c87874e289cf947c1d7f9ded4 (table test-table [id=83b77b7fdcc14ac39886d86c798e1a1c]) successfully deleted
Sep 24 00:56:50 dist-test-slave-mmhs krb5kdc[16029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 24 00:56:50 dist-test-slave-mmhs krb5kdc[16029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:56:50 dist-test-slave-mmhs krb5kdc[16029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
W20260924 00:56:50.411280 16620 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:56:50.399617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34628 (local address 127.8.64.190:36845)
0924 00:56:50.399851 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:50.399861 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:50.400316 (+ 455us) server_negotiation.cc:407] Connection header received
0924 00:56:50.400564 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:50.400568 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:50.400690 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:50.400805 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:50.402552 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:50.403289 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:50.405115 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:50.405517 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:50.405606 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:50.411019 (+ 5413us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34628: BlockingRecv error: recv got EOF from 127.0.0.1:34628 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Sep 24 00:56:50 dist-test-slave-mmhs krb5kdc[16029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM
I20260924 00:56:50.433122 16620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:50.414099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34638 (local address 127.8.64.190:36845)
0924 00:56:50.414317 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:50.414324 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:50.418874 (+ 4550us) server_negotiation.cc:407] Connection header received
0924 00:56:50.419021 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:50.419027 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:50.419108 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:50.419219 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:50.420312 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:50.421476 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:50.422772 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:50.423424 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:50.423552 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:50.427147 (+ 3595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:50.427153 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:50.427158 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:50.427198 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:50.430411 (+ 3213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:50.431176 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:50.431182 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:50.431185 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:50.431334 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:50.431665 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:50.431670 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:50.431673 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:50.431994 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:50.432261 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:50.432554 (+ 293us) server_negotiation.cc:299] Negotiation successful
0924 00:56:50.432780 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260924 00:56:50.437080 16074 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34638:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:56:50.437827 16074 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:56:50.444602 16211 tablet_service.cc:1467] Processing CreateTablet for tablet 121e3ec9b5ab4019840bef683dc5ec47 (DEFAULT_TABLE table=test-table [id=3172ac6516bc43c590b4a073661db888]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:56:50.444877 16349 tablet_service.cc:1467] Processing CreateTablet for tablet 121e3ec9b5ab4019840bef683dc5ec47 (DEFAULT_TABLE table=test-table [id=3172ac6516bc43c590b4a073661db888]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:56:50.445053 16211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 121e3ec9b5ab4019840bef683dc5ec47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:50.445298 16349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 121e3ec9b5ab4019840bef683dc5ec47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:50.449803 16584 tablet_bootstrap.cc:492] T 121e3ec9b5ab4019840bef683dc5ec47 P 456e100a017b491d8d11fd3a68031fd6: Bootstrap starting.
I20260924 00:56:50.450856 16488 tablet_service.cc:1467] Processing CreateTablet for tablet 121e3ec9b5ab4019840bef683dc5ec47 (DEFAULT_TABLE table=test-table [id=3172ac6516bc43c590b4a073661db888]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:56:50.451313 16488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 121e3ec9b5ab4019840bef683dc5ec47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:50.451511 16584 tablet_bootstrap.cc:654] T 121e3ec9b5ab4019840bef683dc5ec47 P 456e100a017b491d8d11fd3a68031fd6: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:50.451512 16585 tablet_bootstrap.cc:492] T 121e3ec9b5ab4019840bef683dc5ec47 P adfab66f8ae74a76ac0d1d09df6a617c: Bootstrap starting.
I20260924 00:56:50.453047 16585 tablet_bootstrap.cc:654] T 121e3ec9b5ab4019840bef683dc5ec47 P adfab66f8ae74a76ac0d1d09df6a617c: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:50.460400 16583 tablet_bootstrap.cc:492] T 121e3ec9b5ab4019840bef683dc5ec47 P bed6c0acef1b447580fffb7275147fb8: Bootstrap starting.
I20260924 00:56:50.462358 16583 tablet_bootstrap.cc:654] T 121e3ec9b5ab4019840bef683dc5ec47 P bed6c0acef1b447580fffb7275147fb8: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:50.465329 16585 tablet_bootstrap.cc:492] T 121e3ec9b5ab4019840bef683dc5ec47 P adfab66f8ae74a76ac0d1d09df6a617c: No bootstrap required, opened a new log
I20260924 00:56:50.465505 16585 ts_tablet_manager.cc:1397] T 121e3ec9b5ab4019840bef683dc5ec47 P adfab66f8ae74a76ac0d1d09df6a617c: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.004s
I20260924 00:56:50.466246 16585 raft_consensus.cc:348] T 121e3ec9b5ab4019840bef683dc5ec47 P adfab66f8ae74a76ac0d1d09df6a617c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } }
I20260924 00:56:50.466558 16585 raft_consensus.cc:374] T 121e3ec9b5ab4019840bef683dc5ec47 P adfab66f8ae74a76ac0d1d09df6a617c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:50.466652 16585 raft_consensus.cc:729] T 121e3ec9b5ab4019840bef683dc5ec47 P adfab66f8ae74a76ac0d1d09df6a617c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adfab66f8ae74a76ac0d1d09df6a617c, State: Initialized, Role: FOLLOWER
I20260924 00:56:50.466914 16585 consensus_queue.cc:260] T 121e3ec9b5ab4019840bef683dc5ec47 P adfab66f8ae74a76ac0d1d09df6a617c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } }
I20260924 00:56:50.470017 16585 ts_tablet_manager.cc:1428] T 121e3ec9b5ab4019840bef683dc5ec47 P adfab66f8ae74a76ac0d1d09df6a617c: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260924 00:56:50.471496 16583 tablet_bootstrap.cc:492] T 121e3ec9b5ab4019840bef683dc5ec47 P bed6c0acef1b447580fffb7275147fb8: No bootstrap required, opened a new log
I20260924 00:56:50.471660 16583 ts_tablet_manager.cc:1397] T 121e3ec9b5ab4019840bef683dc5ec47 P bed6c0acef1b447580fffb7275147fb8: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.001s
I20260924 00:56:50.472604 16583 raft_consensus.cc:348] T 121e3ec9b5ab4019840bef683dc5ec47 P bed6c0acef1b447580fffb7275147fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } }
I20260924 00:56:50.472980 16583 raft_consensus.cc:374] T 121e3ec9b5ab4019840bef683dc5ec47 P bed6c0acef1b447580fffb7275147fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:50.473124 16583 raft_consensus.cc:729] T 121e3ec9b5ab4019840bef683dc5ec47 P bed6c0acef1b447580fffb7275147fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bed6c0acef1b447580fffb7275147fb8, State: Initialized, Role: FOLLOWER
I20260924 00:56:50.473533 16583 consensus_queue.cc:260] T 121e3ec9b5ab4019840bef683dc5ec47 P bed6c0acef1b447580fffb7275147fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } }
I20260924 00:56:50.474355 16583 ts_tablet_manager.cc:1428] T 121e3ec9b5ab4019840bef683dc5ec47 P bed6c0acef1b447580fffb7275147fb8: Time spent starting tablet: real 0.003s user 0.000s sys 0.000s
I20260924 00:56:50.476698 16584 tablet_bootstrap.cc:492] T 121e3ec9b5ab4019840bef683dc5ec47 P 456e100a017b491d8d11fd3a68031fd6: No bootstrap required, opened a new log
I20260924 00:56:50.477700 16584 ts_tablet_manager.cc:1397] T 121e3ec9b5ab4019840bef683dc5ec47 P 456e100a017b491d8d11fd3a68031fd6: Time spent bootstrapping tablet: real 0.028s user 0.013s sys 0.006s
I20260924 00:56:50.478483 16584 raft_consensus.cc:348] T 121e3ec9b5ab4019840bef683dc5ec47 P 456e100a017b491d8d11fd3a68031fd6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } }
I20260924 00:56:50.478708 16584 raft_consensus.cc:374] T 121e3ec9b5ab4019840bef683dc5ec47 P 456e100a017b491d8d11fd3a68031fd6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:50.478821 16584 raft_consensus.cc:729] T 121e3ec9b5ab4019840bef683dc5ec47 P 456e100a017b491d8d11fd3a68031fd6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 456e100a017b491d8d11fd3a68031fd6, State: Initialized, Role: FOLLOWER
I20260924 00:56:50.479056 16584 consensus_queue.cc:260] T 121e3ec9b5ab4019840bef683dc5ec47 P 456e100a017b491d8d11fd3a68031fd6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } }
I20260924 00:56:50.479525 16584 ts_tablet_manager.cc:1428] T 121e3ec9b5ab4019840bef683dc5ec47 P 456e100a017b491d8d11fd3a68031fd6: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260924 00:56:50.586496 16634 raft_consensus.cc:482] T 121e3ec9b5ab4019840bef683dc5ec47 P bed6c0acef1b447580fffb7275147fb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:56:50.586715 16634 raft_consensus.cc:504] T 121e3ec9b5ab4019840bef683dc5ec47 P bed6c0acef1b447580fffb7275147fb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } }
I20260924 00:56:50.587675 16634 leader_election.cc:288] T 121e3ec9b5ab4019840bef683dc5ec47 P bed6c0acef1b447580fffb7275147fb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers adfab66f8ae74a76ac0d1d09df6a617c (127.8.64.129:35697), 456e100a017b491d8d11fd3a68031fd6 (127.8.64.130:45455)
I20260924 00:56:50.588276 16234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "121e3ec9b5ab4019840bef683dc5ec47" candidate_uuid: "bed6c0acef1b447580fffb7275147fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" is_pre_election: true
I20260924 00:56:50.588356 16370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "121e3ec9b5ab4019840bef683dc5ec47" candidate_uuid: "bed6c0acef1b447580fffb7275147fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "456e100a017b491d8d11fd3a68031fd6" is_pre_election: true
I20260924 00:56:50.588542 16234 raft_consensus.cc:2454] T 121e3ec9b5ab4019840bef683dc5ec47 P adfab66f8ae74a76ac0d1d09df6a617c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bed6c0acef1b447580fffb7275147fb8 in term 0.
I20260924 00:56:50.588608 16370 raft_consensus.cc:2454] T 121e3ec9b5ab4019840bef683dc5ec47 P 456e100a017b491d8d11fd3a68031fd6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bed6c0acef1b447580fffb7275147fb8 in term 0.
I20260924 00:56:50.589046 16441 leader_election.cc:302] T 121e3ec9b5ab4019840bef683dc5ec47 P bed6c0acef1b447580fffb7275147fb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 456e100a017b491d8d11fd3a68031fd6, bed6c0acef1b447580fffb7275147fb8; no voters:
I20260924 00:56:50.589408 16634 raft_consensus.cc:2781] T 121e3ec9b5ab4019840bef683dc5ec47 P bed6c0acef1b447580fffb7275147fb8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:56:50.589608 16634 raft_consensus.cc:482] T 121e3ec9b5ab4019840bef683dc5ec47 P bed6c0acef1b447580fffb7275147fb8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:56:50.589745 16634 raft_consensus.cc:3037] T 121e3ec9b5ab4019840bef683dc5ec47 P bed6c0acef1b447580fffb7275147fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:50.593986 16634 raft_consensus.cc:504] T 121e3ec9b5ab4019840bef683dc5ec47 P bed6c0acef1b447580fffb7275147fb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } }
I20260924 00:56:50.595340 16370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "121e3ec9b5ab4019840bef683dc5ec47" candidate_uuid: "bed6c0acef1b447580fffb7275147fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "456e100a017b491d8d11fd3a68031fd6"
I20260924 00:56:50.595343 16234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "121e3ec9b5ab4019840bef683dc5ec47" candidate_uuid: "bed6c0acef1b447580fffb7275147fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adfab66f8ae74a76ac0d1d09df6a617c"
I20260924 00:56:50.595563 16370 raft_consensus.cc:3037] T 121e3ec9b5ab4019840bef683dc5ec47 P 456e100a017b491d8d11fd3a68031fd6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:50.595566 16234 raft_consensus.cc:3037] T 121e3ec9b5ab4019840bef683dc5ec47 P adfab66f8ae74a76ac0d1d09df6a617c [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:50.598662 16370 raft_consensus.cc:2454] T 121e3ec9b5ab4019840bef683dc5ec47 P 456e100a017b491d8d11fd3a68031fd6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bed6c0acef1b447580fffb7275147fb8 in term 1.
I20260924 00:56:50.598887 16234 raft_consensus.cc:2454] T 121e3ec9b5ab4019840bef683dc5ec47 P adfab66f8ae74a76ac0d1d09df6a617c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bed6c0acef1b447580fffb7275147fb8 in term 1.
I20260924 00:56:50.599184 16634 leader_election.cc:288] T 121e3ec9b5ab4019840bef683dc5ec47 P bed6c0acef1b447580fffb7275147fb8 [CANDIDATE]: Term 1 election: Requested vote from peers adfab66f8ae74a76ac0d1d09df6a617c (127.8.64.129:35697), 456e100a017b491d8d11fd3a68031fd6 (127.8.64.130:45455)
I20260924 00:56:50.599285 16441 leader_election.cc:302] T 121e3ec9b5ab4019840bef683dc5ec47 P bed6c0acef1b447580fffb7275147fb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 456e100a017b491d8d11fd3a68031fd6, bed6c0acef1b447580fffb7275147fb8; no voters:
I20260924 00:56:50.599656 16634 raft_consensus.cc:2781] T 121e3ec9b5ab4019840bef683dc5ec47 P bed6c0acef1b447580fffb7275147fb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:56:50.599822 16634 raft_consensus.cc:686] T 121e3ec9b5ab4019840bef683dc5ec47 P bed6c0acef1b447580fffb7275147fb8 [term 1 LEADER]: Becoming Leader. State: Replica: bed6c0acef1b447580fffb7275147fb8, State: Running, Role: LEADER
I20260924 00:56:50.600208 16634 consensus_queue.cc:237] T 121e3ec9b5ab4019840bef683dc5ec47 P bed6c0acef1b447580fffb7275147fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } }
I20260924 00:56:50.604060 16073 catalog_manager.cc:5495] T 121e3ec9b5ab4019840bef683dc5ec47 P bed6c0acef1b447580fffb7275147fb8 reported cstate change: term changed from 0 to 1, leader changed from <none> to bed6c0acef1b447580fffb7275147fb8 (127.8.64.131). New cstate: current_term: 1 leader_uuid: "bed6c0acef1b447580fffb7275147fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } health_report { overall_health: HEALTHY } } }
I20260924 00:56:50.630919 16631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:50.623631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55998 (local address 127.8.64.131:36111)
0924 00:56:50.623802 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:50.623809 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:50.624110 (+ 301us) server_negotiation.cc:407] Connection header received
0924 00:56:50.624349 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:50.624355 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:50.624476 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:50.624573 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:56:50.625036 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:50.626210 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:50.628587 (+ 2377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:50.629260 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:50.629382 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:50.630024 (+ 642us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:56:50.630166 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:56:50.630265 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:50.630475 (+ 210us) server_negotiation.cc:299] Negotiation successful
0924 00:56:50.630574 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260924 00:56:50.634860 16370 raft_consensus.cc:1261] T 121e3ec9b5ab4019840bef683dc5ec47 P 456e100a017b491d8d11fd3a68031fd6 [term 1 FOLLOWER]: Refusing update from remote peer bed6c0acef1b447580fffb7275147fb8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:56:50.634884 16234 raft_consensus.cc:1261] T 121e3ec9b5ab4019840bef683dc5ec47 P adfab66f8ae74a76ac0d1d09df6a617c [term 1 FOLLOWER]: Refusing update from remote peer bed6c0acef1b447580fffb7275147fb8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:56:50.635538 16589 consensus_queue.cc:1035] T 121e3ec9b5ab4019840bef683dc5ec47 P bed6c0acef1b447580fffb7275147fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:56:50.636050 16626 consensus_queue.cc:1035] T 121e3ec9b5ab4019840bef683dc5ec47 P bed6c0acef1b447580fffb7275147fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:56:50.653924 16074 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:56:50.654224 16074 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:56:50.659274 16074 catalog_manager.cc:5782] T 00000000000000000000000000000000 P debc27d99cb84de7940c1a40cfe8fff2: Sending DeleteTablet for 3 replicas of tablet 121e3ec9b5ab4019840bef683dc5ec47
I20260924 00:56:50.660194 16349 tablet_service.cc:1514] Processing DeleteTablet for tablet 121e3ec9b5ab4019840bef683dc5ec47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:56:50 UTC) from {username='oryx', principal='oryx/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:53460
I20260924 00:56:50.660517 16646 tablet_replica.cc:330] T 121e3ec9b5ab4019840bef683dc5ec47 P 456e100a017b491d8d11fd3a68031fd6: stopping tablet replica
I20260924 00:56:50.660475 16211 tablet_service.cc:1514] Processing DeleteTablet for tablet 121e3ec9b5ab4019840bef683dc5ec47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:56:50 UTC) from {username='oryx', principal='oryx/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:60358
I20260924 00:56:50.660713 16646 raft_consensus.cc:2229] T 121e3ec9b5ab4019840bef683dc5ec47 P 456e100a017b491d8d11fd3a68031fd6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:56:50.660748 16210 tablet_service.cc:1514] Processing DeleteTablet for tablet 121e3ec9b5ab4019840bef683dc5ec47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:56:50 UTC) from {username='oryx', principal='oryx/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:60358
I20260924 00:56:50.660966 16644 tablet_replica.cc:330] T 121e3ec9b5ab4019840bef683dc5ec47 P adfab66f8ae74a76ac0d1d09df6a617c: stopping tablet replica
I20260924 00:56:50.660964 16646 raft_consensus.cc:2258] T 121e3ec9b5ab4019840bef683dc5ec47 P 456e100a017b491d8d11fd3a68031fd6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:56:50.661140 16644 raft_consensus.cc:2229] T 121e3ec9b5ab4019840bef683dc5ec47 P adfab66f8ae74a76ac0d1d09df6a617c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:56:50.661347 16644 raft_consensus.cc:2258] T 121e3ec9b5ab4019840bef683dc5ec47 P adfab66f8ae74a76ac0d1d09df6a617c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:56:50.661991 16646 ts_tablet_manager.cc:1905] T 121e3ec9b5ab4019840bef683dc5ec47 P 456e100a017b491d8d11fd3a68031fd6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:56:50.662060 16488 tablet_service.cc:1514] Processing DeleteTablet for tablet 121e3ec9b5ab4019840bef683dc5ec47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:56:50 UTC) from {username='oryx', principal='oryx/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:55990
I20260924 00:56:50.661935 16074 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34638:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:56:50.662338 16645 tablet_replica.cc:330] T 121e3ec9b5ab4019840bef683dc5ec47 P bed6c0acef1b447580fffb7275147fb8: stopping tablet replica
I20260924 00:56:50.662530 16645 raft_consensus.cc:2229] T 121e3ec9b5ab4019840bef683dc5ec47 P bed6c0acef1b447580fffb7275147fb8 [term 1 LEADER]: Raft consensus shutting down.
W20260924 00:56:50.662549 16074 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:56:50.662971 16645 raft_consensus.cc:2258] T 121e3ec9b5ab4019840bef683dc5ec47 P bed6c0acef1b447580fffb7275147fb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:56:50.666210 16645 ts_tablet_manager.cc:1905] T 121e3ec9b5ab4019840bef683dc5ec47 P bed6c0acef1b447580fffb7275147fb8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:56:50.668587 16646 ts_tablet_manager.cc:1918] T 121e3ec9b5ab4019840bef683dc5ec47 P 456e100a017b491d8d11fd3a68031fd6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:56:50.668732 16646 log.cc:1196] T 121e3ec9b5ab4019840bef683dc5ec47 P 456e100a017b491d8d11fd3a68031fd6: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-1/wal/wals/121e3ec9b5ab4019840bef683dc5ec47
I20260924 00:56:50.669221 16646 ts_tablet_manager.cc:1939] T 121e3ec9b5ab4019840bef683dc5ec47 P 456e100a017b491d8d11fd3a68031fd6: Deleting consensus metadata
I20260924 00:56:50.670389 16059 catalog_manager.cc:4841] TS 456e100a017b491d8d11fd3a68031fd6 (127.8.64.130:45455): tablet 121e3ec9b5ab4019840bef683dc5ec47 (table test-table [id=3172ac6516bc43c590b4a073661db888]) successfully deleted
I20260924 00:56:50.675004 16349 tablet_service.cc:1467] Processing CreateTablet for tablet 3524564701594b1e9284354fb09ae9d7 (DEFAULT_TABLE table=test-table [id=b6309618056a438b8d2fae015832629d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:56:50.675401 16349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3524564701594b1e9284354fb09ae9d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:50.677300 16645 ts_tablet_manager.cc:1918] T 121e3ec9b5ab4019840bef683dc5ec47 P bed6c0acef1b447580fffb7275147fb8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:56:50.677469 16645 log.cc:1196] T 121e3ec9b5ab4019840bef683dc5ec47 P bed6c0acef1b447580fffb7275147fb8: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-2/wal/wals/121e3ec9b5ab4019840bef683dc5ec47
I20260924 00:56:50.677935 16645 ts_tablet_manager.cc:1939] T 121e3ec9b5ab4019840bef683dc5ec47 P bed6c0acef1b447580fffb7275147fb8: Deleting consensus metadata
I20260924 00:56:50.679625 16644 ts_tablet_manager.cc:1905] T 121e3ec9b5ab4019840bef683dc5ec47 P adfab66f8ae74a76ac0d1d09df6a617c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:56:50.681633 16058 catalog_manager.cc:4841] TS bed6c0acef1b447580fffb7275147fb8 (127.8.64.131:36111): tablet 121e3ec9b5ab4019840bef683dc5ec47 (table test-table [id=3172ac6516bc43c590b4a073661db888]) successfully deleted
I20260924 00:56:50.682847 16210 tablet_service.cc:1467] Processing CreateTablet for tablet 3524564701594b1e9284354fb09ae9d7 (DEFAULT_TABLE table=test-table [id=b6309618056a438b8d2fae015832629d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:56:50.683265 16210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3524564701594b1e9284354fb09ae9d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:50.686426 16488 tablet_service.cc:1467] Processing CreateTablet for tablet 3524564701594b1e9284354fb09ae9d7 (DEFAULT_TABLE table=test-table [id=b6309618056a438b8d2fae015832629d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:56:50.686857 16488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3524564701594b1e9284354fb09ae9d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:50.687842 16644 ts_tablet_manager.cc:1918] T 121e3ec9b5ab4019840bef683dc5ec47 P adfab66f8ae74a76ac0d1d09df6a617c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:56:50.687990 16644 log.cc:1196] T 121e3ec9b5ab4019840bef683dc5ec47 P adfab66f8ae74a76ac0d1d09df6a617c: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-0/wal/wals/121e3ec9b5ab4019840bef683dc5ec47
I20260924 00:56:50.688426 16644 ts_tablet_manager.cc:1939] T 121e3ec9b5ab4019840bef683dc5ec47 P adfab66f8ae74a76ac0d1d09df6a617c: Deleting consensus metadata
I20260924 00:56:50.690405 16058 catalog_manager.cc:4841] TS adfab66f8ae74a76ac0d1d09df6a617c (127.8.64.129:35697): tablet 121e3ec9b5ab4019840bef683dc5ec47 (table test-table [id=3172ac6516bc43c590b4a073661db888]) successfully deleted
I20260924 00:56:50.693243 16585 tablet_bootstrap.cc:492] T 3524564701594b1e9284354fb09ae9d7 P adfab66f8ae74a76ac0d1d09df6a617c: Bootstrap starting.
W20260924 00:56:50.693451 16058 catalog_manager.cc:4808] TS adfab66f8ae74a76ac0d1d09df6a617c (127.8.64.129:35697): delete failed for tablet 121e3ec9b5ab4019840bef683dc5ec47 because the tablet was not found. No further retry: Not found: Tablet not found: 121e3ec9b5ab4019840bef683dc5ec47
I20260924 00:56:50.695477 16585 tablet_bootstrap.cc:654] T 3524564701594b1e9284354fb09ae9d7 P adfab66f8ae74a76ac0d1d09df6a617c: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:50.697185 16584 tablet_bootstrap.cc:492] T 3524564701594b1e9284354fb09ae9d7 P 456e100a017b491d8d11fd3a68031fd6: Bootstrap starting.
I20260924 00:56:50.699304 16584 tablet_bootstrap.cc:654] T 3524564701594b1e9284354fb09ae9d7 P 456e100a017b491d8d11fd3a68031fd6: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:50.702217 16583 tablet_bootstrap.cc:492] T 3524564701594b1e9284354fb09ae9d7 P bed6c0acef1b447580fffb7275147fb8: Bootstrap starting.
I20260924 00:56:50.704507 16583 tablet_bootstrap.cc:654] T 3524564701594b1e9284354fb09ae9d7 P bed6c0acef1b447580fffb7275147fb8: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:50.723356 16583 tablet_bootstrap.cc:492] T 3524564701594b1e9284354fb09ae9d7 P bed6c0acef1b447580fffb7275147fb8: No bootstrap required, opened a new log
I20260924 00:56:50.723562 16583 ts_tablet_manager.cc:1397] T 3524564701594b1e9284354fb09ae9d7 P bed6c0acef1b447580fffb7275147fb8: Time spent bootstrapping tablet: real 0.021s user 0.011s sys 0.000s
I20260924 00:56:50.724705 16583 raft_consensus.cc:348] T 3524564701594b1e9284354fb09ae9d7 P bed6c0acef1b447580fffb7275147fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } }
I20260924 00:56:50.725036 16583 raft_consensus.cc:374] T 3524564701594b1e9284354fb09ae9d7 P bed6c0acef1b447580fffb7275147fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:50.725134 16583 raft_consensus.cc:729] T 3524564701594b1e9284354fb09ae9d7 P bed6c0acef1b447580fffb7275147fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bed6c0acef1b447580fffb7275147fb8, State: Initialized, Role: FOLLOWER
I20260924 00:56:50.725698 16583 consensus_queue.cc:260] T 3524564701594b1e9284354fb09ae9d7 P bed6c0acef1b447580fffb7275147fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } }
I20260924 00:56:50.726337 16583 ts_tablet_manager.cc:1428] T 3524564701594b1e9284354fb09ae9d7 P bed6c0acef1b447580fffb7275147fb8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 00:56:50.727813 16584 tablet_bootstrap.cc:492] T 3524564701594b1e9284354fb09ae9d7 P 456e100a017b491d8d11fd3a68031fd6: No bootstrap required, opened a new log
I20260924 00:56:50.728039 16584 ts_tablet_manager.cc:1397] T 3524564701594b1e9284354fb09ae9d7 P 456e100a017b491d8d11fd3a68031fd6: Time spent bootstrapping tablet: real 0.031s user 0.003s sys 0.009s
I20260924 00:56:50.728792 16584 raft_consensus.cc:348] T 3524564701594b1e9284354fb09ae9d7 P 456e100a017b491d8d11fd3a68031fd6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } }
I20260924 00:56:50.729074 16584 raft_consensus.cc:374] T 3524564701594b1e9284354fb09ae9d7 P 456e100a017b491d8d11fd3a68031fd6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:50.729168 16584 raft_consensus.cc:729] T 3524564701594b1e9284354fb09ae9d7 P 456e100a017b491d8d11fd3a68031fd6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 456e100a017b491d8d11fd3a68031fd6, State: Initialized, Role: FOLLOWER
I20260924 00:56:50.729439 16584 consensus_queue.cc:260] T 3524564701594b1e9284354fb09ae9d7 P 456e100a017b491d8d11fd3a68031fd6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } }
I20260924 00:56:50.730509 16584 ts_tablet_manager.cc:1428] T 3524564701594b1e9284354fb09ae9d7 P 456e100a017b491d8d11fd3a68031fd6: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260924 00:56:50.731704 16585 tablet_bootstrap.cc:492] T 3524564701594b1e9284354fb09ae9d7 P adfab66f8ae74a76ac0d1d09df6a617c: No bootstrap required, opened a new log
I20260924 00:56:50.731891 16585 ts_tablet_manager.cc:1397] T 3524564701594b1e9284354fb09ae9d7 P adfab66f8ae74a76ac0d1d09df6a617c: Time spent bootstrapping tablet: real 0.039s user 0.005s sys 0.018s
I20260924 00:56:50.732690 16585 raft_consensus.cc:348] T 3524564701594b1e9284354fb09ae9d7 P adfab66f8ae74a76ac0d1d09df6a617c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } }
I20260924 00:56:50.732978 16585 raft_consensus.cc:374] T 3524564701594b1e9284354fb09ae9d7 P adfab66f8ae74a76ac0d1d09df6a617c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:50.733063 16585 raft_consensus.cc:729] T 3524564701594b1e9284354fb09ae9d7 P adfab66f8ae74a76ac0d1d09df6a617c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adfab66f8ae74a76ac0d1d09df6a617c, State: Initialized, Role: FOLLOWER
I20260924 00:56:50.733307 16585 consensus_queue.cc:260] T 3524564701594b1e9284354fb09ae9d7 P adfab66f8ae74a76ac0d1d09df6a617c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } }
I20260924 00:56:50.733807 16585 ts_tablet_manager.cc:1428] T 3524564701594b1e9284354fb09ae9d7 P adfab66f8ae74a76ac0d1d09df6a617c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260924 00:56:50.743360 16594 raft_consensus.cc:482] T 3524564701594b1e9284354fb09ae9d7 P 456e100a017b491d8d11fd3a68031fd6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:56:50.743547 16594 raft_consensus.cc:504] T 3524564701594b1e9284354fb09ae9d7 P 456e100a017b491d8d11fd3a68031fd6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } }
I20260924 00:56:50.744141 16594 leader_election.cc:288] T 3524564701594b1e9284354fb09ae9d7 P 456e100a017b491d8d11fd3a68031fd6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers adfab66f8ae74a76ac0d1d09df6a617c (127.8.64.129:35697), bed6c0acef1b447580fffb7275147fb8 (127.8.64.131:36111)
I20260924 00:56:50.744709 16234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3524564701594b1e9284354fb09ae9d7" candidate_uuid: "456e100a017b491d8d11fd3a68031fd6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" is_pre_election: true
I20260924 00:56:50.744733 16514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3524564701594b1e9284354fb09ae9d7" candidate_uuid: "456e100a017b491d8d11fd3a68031fd6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bed6c0acef1b447580fffb7275147fb8" is_pre_election: true
I20260924 00:56:50.744941 16234 raft_consensus.cc:2454] T 3524564701594b1e9284354fb09ae9d7 P adfab66f8ae74a76ac0d1d09df6a617c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 456e100a017b491d8d11fd3a68031fd6 in term 0.
I20260924 00:56:50.744979 16514 raft_consensus.cc:2454] T 3524564701594b1e9284354fb09ae9d7 P bed6c0acef1b447580fffb7275147fb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 456e100a017b491d8d11fd3a68031fd6 in term 0.
I20260924 00:56:50.745491 16301 leader_election.cc:302] T 3524564701594b1e9284354fb09ae9d7 P 456e100a017b491d8d11fd3a68031fd6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 456e100a017b491d8d11fd3a68031fd6, adfab66f8ae74a76ac0d1d09df6a617c; no voters:
I20260924 00:56:50.745926 16594 raft_consensus.cc:2781] T 3524564701594b1e9284354fb09ae9d7 P 456e100a017b491d8d11fd3a68031fd6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:56:50.746011 16594 raft_consensus.cc:482] T 3524564701594b1e9284354fb09ae9d7 P 456e100a017b491d8d11fd3a68031fd6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:56:50.746095 16594 raft_consensus.cc:3037] T 3524564701594b1e9284354fb09ae9d7 P 456e100a017b491d8d11fd3a68031fd6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:50.748991 16594 raft_consensus.cc:504] T 3524564701594b1e9284354fb09ae9d7 P 456e100a017b491d8d11fd3a68031fd6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } }
I20260924 00:56:50.750139 16514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3524564701594b1e9284354fb09ae9d7" candidate_uuid: "456e100a017b491d8d11fd3a68031fd6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bed6c0acef1b447580fffb7275147fb8"
I20260924 00:56:50.750392 16514 raft_consensus.cc:3037] T 3524564701594b1e9284354fb09ae9d7 P bed6c0acef1b447580fffb7275147fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:50.750447 16234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3524564701594b1e9284354fb09ae9d7" candidate_uuid: "456e100a017b491d8d11fd3a68031fd6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adfab66f8ae74a76ac0d1d09df6a617c"
I20260924 00:56:50.750622 16234 raft_consensus.cc:3037] T 3524564701594b1e9284354fb09ae9d7 P adfab66f8ae74a76ac0d1d09df6a617c [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:50.753490 16514 raft_consensus.cc:2454] T 3524564701594b1e9284354fb09ae9d7 P bed6c0acef1b447580fffb7275147fb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 456e100a017b491d8d11fd3a68031fd6 in term 1.
I20260924 00:56:50.753482 16234 raft_consensus.cc:2454] T 3524564701594b1e9284354fb09ae9d7 P adfab66f8ae74a76ac0d1d09df6a617c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 456e100a017b491d8d11fd3a68031fd6 in term 1.
I20260924 00:56:50.753880 16301 leader_election.cc:302] T 3524564701594b1e9284354fb09ae9d7 P 456e100a017b491d8d11fd3a68031fd6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 456e100a017b491d8d11fd3a68031fd6, adfab66f8ae74a76ac0d1d09df6a617c; no voters:
I20260924 00:56:50.754230 16602 raft_consensus.cc:2781] T 3524564701594b1e9284354fb09ae9d7 P 456e100a017b491d8d11fd3a68031fd6 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:56:50.754470 16602 raft_consensus.cc:686] T 3524564701594b1e9284354fb09ae9d7 P 456e100a017b491d8d11fd3a68031fd6 [term 1 LEADER]: Becoming Leader. State: Replica: 456e100a017b491d8d11fd3a68031fd6, State: Running, Role: LEADER
I20260924 00:56:50.754726 16602 consensus_queue.cc:237] T 3524564701594b1e9284354fb09ae9d7 P 456e100a017b491d8d11fd3a68031fd6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } }
I20260924 00:56:50.756305 16594 leader_election.cc:288] T 3524564701594b1e9284354fb09ae9d7 P 456e100a017b491d8d11fd3a68031fd6 [CANDIDATE]: Term 1 election: Requested vote from peers adfab66f8ae74a76ac0d1d09df6a617c (127.8.64.129:35697), bed6c0acef1b447580fffb7275147fb8 (127.8.64.131:36111)
I20260924 00:56:50.772997 16073 catalog_manager.cc:5495] T 3524564701594b1e9284354fb09ae9d7 P 456e100a017b491d8d11fd3a68031fd6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 456e100a017b491d8d11fd3a68031fd6 (127.8.64.130). New cstate: current_term: 1 leader_uuid: "456e100a017b491d8d11fd3a68031fd6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "456e100a017b491d8d11fd3a68031fd6" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 45455 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 } health_report { overall_health: UNKNOWN } } }
I20260924 00:56:50.814021 16670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:50.806330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53478 (local address 127.8.64.130:45455)
0924 00:56:50.807341 (+ 1011us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:50.807349 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:50.807370 (+ 21us) server_negotiation.cc:407] Connection header received
0924 00:56:50.807439 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:50.807446 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:50.807567 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:50.807675 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:56:50.808118 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:50.809389 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:50.811437 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:50.812337 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:50.812527 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:50.813094 (+ 567us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:56:50.813252 (+ 158us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:56:50.813384 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:50.813506 (+ 122us) server_negotiation.cc:299] Negotiation successful
0924 00:56:50.813604 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":872,"thread_start_us":718,"threads_started":1}
I20260924 00:56:50.827191 16234 raft_consensus.cc:1261] T 3524564701594b1e9284354fb09ae9d7 P adfab66f8ae74a76ac0d1d09df6a617c [term 1 FOLLOWER]: Refusing update from remote peer 456e100a017b491d8d11fd3a68031fd6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:56:50.827447 16514 raft_consensus.cc:1261] T 3524564701594b1e9284354fb09ae9d7 P bed6c0acef1b447580fffb7275147fb8 [term 1 FOLLOWER]: Refusing update from remote peer 456e100a017b491d8d11fd3a68031fd6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:56:50.828018 16594 consensus_queue.cc:1035] T 3524564701594b1e9284354fb09ae9d7 P 456e100a017b491d8d11fd3a68031fd6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "adfab66f8ae74a76ac0d1d09df6a617c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:56:50.828466 16602 consensus_queue.cc:1035] T 3524564701594b1e9284354fb09ae9d7 P 456e100a017b491d8d11fd3a68031fd6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bed6c0acef1b447580fffb7275147fb8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:56:50.913379 16073 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:56:50.913641 16073 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:56:50.917320 16073 catalog_manager.cc:5782] T 00000000000000000000000000000000 P debc27d99cb84de7940c1a40cfe8fff2: Sending DeleteTablet for 3 replicas of tablet 3524564701594b1e9284354fb09ae9d7
I20260924 00:56:50.918046 16210 tablet_service.cc:1514] Processing DeleteTablet for tablet 3524564701594b1e9284354fb09ae9d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:56:50 UTC) from {username='oryx', principal='oryx/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:60358
I20260924 00:56:50.918560 16644 tablet_replica.cc:330] T 3524564701594b1e9284354fb09ae9d7 P adfab66f8ae74a76ac0d1d09df6a617c: stopping tablet replica
I20260924 00:56:50.918532 16488 tablet_service.cc:1514] Processing DeleteTablet for tablet 3524564701594b1e9284354fb09ae9d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:56:50 UTC) from {username='oryx', principal='oryx/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:55990
I20260924 00:56:50.918730 16644 raft_consensus.cc:2229] T 3524564701594b1e9284354fb09ae9d7 P adfab66f8ae74a76ac0d1d09df6a617c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:56:50.918829 16645 tablet_replica.cc:330] T 3524564701594b1e9284354fb09ae9d7 P bed6c0acef1b447580fffb7275147fb8: stopping tablet replica
I20260924 00:56:50.918931 16644 raft_consensus.cc:2258] T 3524564701594b1e9284354fb09ae9d7 P adfab66f8ae74a76ac0d1d09df6a617c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:56:50.919003 16645 raft_consensus.cc:2229] T 3524564701594b1e9284354fb09ae9d7 P bed6c0acef1b447580fffb7275147fb8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:56:50.919195 16645 raft_consensus.cc:2258] T 3524564701594b1e9284354fb09ae9d7 P bed6c0acef1b447580fffb7275147fb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:56:50.919195 16349 tablet_service.cc:1514] Processing DeleteTablet for tablet 3524564701594b1e9284354fb09ae9d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:56:50 UTC) from {username='oryx', principal='oryx/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:53460
I20260924 00:56:50.919617 16646 tablet_replica.cc:330] T 3524564701594b1e9284354fb09ae9d7 P 456e100a017b491d8d11fd3a68031fd6: stopping tablet replica
I20260924 00:56:50.919797 16646 raft_consensus.cc:2229] T 3524564701594b1e9284354fb09ae9d7 P 456e100a017b491d8d11fd3a68031fd6 [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:56:50.920095 16646 raft_consensus.cc:2258] T 3524564701594b1e9284354fb09ae9d7 P 456e100a017b491d8d11fd3a68031fd6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:56:50.920885 16645 ts_tablet_manager.cc:1905] T 3524564701594b1e9284354fb09ae9d7 P bed6c0acef1b447580fffb7275147fb8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:56:50.921346 16646 ts_tablet_manager.cc:1905] T 3524564701594b1e9284354fb09ae9d7 P 456e100a017b491d8d11fd3a68031fd6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:56:50.921402 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 16145
I20260924 00:56:50.923098 16644 ts_tablet_manager.cc:1905] T 3524564701594b1e9284354fb09ae9d7 P adfab66f8ae74a76ac0d1d09df6a617c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:56:50.929895 16644 ts_tablet_manager.cc:1918] T 3524564701594b1e9284354fb09ae9d7 P adfab66f8ae74a76ac0d1d09df6a617c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 00:56:50.930048 16644 log.cc:1196] T 3524564701594b1e9284354fb09ae9d7 P adfab66f8ae74a76ac0d1d09df6a617c: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-0/wal/wals/3524564701594b1e9284354fb09ae9d7
I20260924 00:56:50.937873 16646 ts_tablet_manager.cc:1918] T 3524564701594b1e9284354fb09ae9d7 P 456e100a017b491d8d11fd3a68031fd6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 00:56:50.938035 16646 log.cc:1196] T 3524564701594b1e9284354fb09ae9d7 P 456e100a017b491d8d11fd3a68031fd6: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-1/wal/wals/3524564701594b1e9284354fb09ae9d7
I20260924 00:56:50.938575 16646 ts_tablet_manager.cc:1939] T 3524564701594b1e9284354fb09ae9d7 P 456e100a017b491d8d11fd3a68031fd6: Deleting consensus metadata
I20260924 00:56:50.939904 16059 catalog_manager.cc:4841] TS 456e100a017b491d8d11fd3a68031fd6 (127.8.64.130:45455): tablet 3524564701594b1e9284354fb09ae9d7 (table test-table [id=b6309618056a438b8d2fae015832629d]) successfully deleted
I20260924 00:56:50.940428 16644 ts_tablet_manager.cc:1939] T 3524564701594b1e9284354fb09ae9d7 P adfab66f8ae74a76ac0d1d09df6a617c: Deleting consensus metadata
I20260924 00:56:50.941609 16058 catalog_manager.cc:4841] TS adfab66f8ae74a76ac0d1d09df6a617c (127.8.64.129:35697): tablet 3524564701594b1e9284354fb09ae9d7 (table test-table [id=b6309618056a438b8d2fae015832629d]) successfully deleted
I20260924 00:56:50.943641 16645 ts_tablet_manager.cc:1918] T 3524564701594b1e9284354fb09ae9d7 P bed6c0acef1b447580fffb7275147fb8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 00:56:50.943810 16645 log.cc:1196] T 3524564701594b1e9284354fb09ae9d7 P bed6c0acef1b447580fffb7275147fb8: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790211343099518-8450-0/minicluster-data/ts-2/wal/wals/3524564701594b1e9284354fb09ae9d7
I20260924 00:56:50.944331 16645 ts_tablet_manager.cc:1939] T 3524564701594b1e9284354fb09ae9d7 P bed6c0acef1b447580fffb7275147fb8: Deleting consensus metadata
I20260924 00:56:50.945501 16058 catalog_manager.cc:4841] TS bed6c0acef1b447580fffb7275147fb8 (127.8.64.131:36111): tablet 3524564701594b1e9284354fb09ae9d7 (table test-table [id=b6309618056a438b8d2fae015832629d]) successfully deleted
Sep 24 00:56:50 dist-test-slave-mmhs krb5kdc[16029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.64.129@KRBTEST.COM
I20260924 00:56:50.958554 16677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:50.927083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60368 (local address 127.8.64.129:35697)
0924 00:56:50.943250 (+ 16167us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:50.943256 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:50.943275 (+ 19us) server_negotiation.cc:407] Connection header received
0924 00:56:50.943346 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:50.943353 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:50.943437 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:50.943542 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:50.945593 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:50.946612 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:50.947911 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:50.948343 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:50.948479 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:50.952400 (+ 3921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:50.952415 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:50.952430 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:50.952466 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:50.955748 (+ 3282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:50.956439 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:50.956447 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:50.956449 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:50.956567 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:50.956968 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:50.956972 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:50.956974 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:50.957308 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:50.957606 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:50.957854 (+ 248us) server_negotiation.cc:299] Negotiation successful
0924 00:56:50.958122 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16050,"thread_start_us":18775,"threads_started":1}
I20260924 00:56:50.959242 16276 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:56:51.057746 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 16145
I20260924 00:56:51.082880 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 16285
Sep 24 00:56:51 dist-test-slave-mmhs krb5kdc[16029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.64.130@KRBTEST.COM
I20260924 00:56:51.100410 16670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:51.083792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53480 (local address 127.8.64.130:45455)
0924 00:56:51.083957 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:51.083961 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:51.084023 (+ 62us) server_negotiation.cc:407] Connection header received
0924 00:56:51.084217 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:51.084221 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:51.084283 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:51.084369 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:51.086015 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:51.086864 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:51.088687 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:51.089062 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:51.089174 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:51.094501 (+ 5327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:51.094524 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:51.094539 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:51.094574 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:51.097454 (+ 2880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:51.098122 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:51.098131 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:51.098135 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:51.098274 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:51.098673 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:51.098678 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:51.098682 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:51.099052 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:51.099318 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:51.099742 (+ 424us) server_negotiation.cc:299] Negotiation successful
0924 00:56:51.099951 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260924 00:56:51.101205 16415 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:56:51.247576 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 16285
I20260924 00:56:51.269934 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 16424
Sep 24 00:56:51 dist-test-slave-mmhs krb5kdc[16029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.64.131@KRBTEST.COM
I20260924 00:56:51.281423 16681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:51.280459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:35697 (local address 127.8.64.131:34293)
0924 00:56:51.281013 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:51.281180 (+ 167us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.129:35697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":259,"threads_started":1}
W20260924 00:56:51.281706 16440 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.64.129:35697: connect: Connection refused (error 111)
I20260924 00:56:51.283360 16681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:51.282935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:35697 (local address 127.8.64.131:35621)
0924 00:56:51.283110 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:51.283192 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.129:35697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260924 00:56:51.283650 16440 consensus_peers.cc:487] T ef402deffb6b447196b26dc780ce0846 P bed6c0acef1b447580fffb7275147fb8 -> Peer adfab66f8ae74a76ac0d1d09df6a617c (127.8.64.129:35697): Couldn't send request to peer adfab66f8ae74a76ac0d1d09df6a617c. Status: Network error: Client connection negotiation failed: client connection to 127.8.64.129:35697: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 00:56:51.288596 16680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:51.270760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56002 (local address 127.8.64.131:36111)
0924 00:56:51.271360 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:51.271365 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:51.271382 (+ 17us) server_negotiation.cc:407] Connection header received
0924 00:56:51.271437 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:51.271440 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:51.271502 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:51.271593 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:51.273221 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:51.273992 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:51.275938 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:51.276395 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:51.276487 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:51.282673 (+ 6186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:51.282685 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:51.282707 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:51.282754 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:51.285976 (+ 3222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:51.286707 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:51.286713 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:51.286716 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:51.286868 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:51.287241 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:51.287244 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:51.287246 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:51.287569 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:51.287771 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:51.288084 (+ 313us) server_negotiation.cc:299] Negotiation successful
0924 00:56:51.288320 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":314,"threads_started":1}
I20260924 00:56:51.289230 16554 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:56:51.428820 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 16424
I20260924 00:56:51.449715 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 16042
I20260924 00:56:51.451004 16114 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:56:51.564975 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 16042
2026-09-24T00:56:51Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (5751 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790211343099518-8450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:56:51 dist-test-slave-mmhs krb5kdc[16687](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:56:51 dist-test-slave-mmhs krb5kdc[16687](info): set up 2 sockets
Sep 24 00:56:51 dist-test-slave-mmhs krb5kdc[16687](info): commencing operation
krb5kdc: starting...
W20260924 00:56:52.673061 8450 mini_kdc.cc:121] Time spent starting KDC: real 1.062s user 0.005s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Sep 24 00:56:52 dist-test-slave-mmhs krb5kdc[16687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:56:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:56:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.64.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
I20260924 00:56:52.907649 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:42777
--webserver_interface=127.8.64.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:40881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:42777
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:56:53.228711 16701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:53.228997 16701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:53.229164 16701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:53.236291 16701 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:56:53.236395 16701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:53.236502 16701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:53.236599 16701 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:56:53.236685 16701 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:56:53.246411 16701 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:40881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:42777
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:42777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:53.247763 16701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:53.249078 16701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:53.261055 16707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:53.261397 16706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:53.261786 16709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:53.262158 16701 server_base.cc:1034] running on GCE node
I20260924 00:56:53.262926 16701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:53.264276 16701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:53.265484 16701 hybrid_clock.cc:648] HybridClock initialized: now 1790211413265417 us; error 50 us; skew 500 ppm
I20260924 00:56:53.265836 16701 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-24T00:56:53Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1768 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:56:53 dist-test-slave-mmhs krb5kdc[16715](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:56:53 dist-test-slave-mmhs krb5kdc[16715](info): set up 2 sockets
Sep 24 00:56:53 dist-test-slave-mmhs krb5kdc[16715](info): commencing operation
krb5kdc: starting...
W20260924 00:56:54.432968 8450 mini_kdc.cc:121] Time spent starting KDC: real 1.054s user 0.004s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Sep 24 00:56:54 dist-test-slave-mmhs krb5kdc[16715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211414, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:56:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:56:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.64.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
I20260924 00:56:54.661962 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:42507
--webserver_interface=127.8.64.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:46155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:42507
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:56:54.981719 16728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:54.981999 16728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:54.982146 16728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:54.989168 16728 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:56:54.989264 16728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:54.989373 16728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:54.989488 16728 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:56:54.989576 16728 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:56:54.999199 16728 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:46155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:42507
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:42507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:55.000476 16728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:55.001758 16728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:55.013842 16733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:55.013842 16734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:55.013906 16736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:55.014969 16728 server_base.cc:1034] running on GCE node
I20260924 00:56:55.015650 16728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:55.016934 16728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:55.018128 16728 hybrid_clock.cc:648] HybridClock initialized: now 1790211415018079 us; error 44 us; skew 500 ppm
I20260924 00:56:55.018483 16728 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:55 dist-test-slave-mmhs krb5kdc[16715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211415, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:55.027736 16728 init.cc:377] Logged in from keytab as kudu/127.8.64.190@KRBTEST.COM (short username kudu)
I20260924 00:56:55.029247 16728 webserver.cc:460] Webserver started at http://127.8.64.190:44245/ using document root <none> and password file <none>
I20260924 00:56:55.029934 16728 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:55.030076 16728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:55.030448 16728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:56:55.033337 16728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "3a538f6d12a64a48a45a88dc8773ab3f"
format_stamp: "Formatted at 2026-09-24 00:56:55 on dist-test-slave-mmhs"
server_key: "fe89985d82180ea1fba0b470a9cccc05"
server_key_iv: "bba8a335b756bc051e42ffb6babedd41"
server_key_version: "encryptionkey@0"
I20260924 00:56:55.034116 16728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "3a538f6d12a64a48a45a88dc8773ab3f"
format_stamp: "Formatted at 2026-09-24 00:56:55 on dist-test-slave-mmhs"
server_key: "fe89985d82180ea1fba0b470a9cccc05"
server_key_iv: "bba8a335b756bc051e42ffb6babedd41"
server_key_version: "encryptionkey@0"
I20260924 00:56:55.039016 16728 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260924 00:56:55.042255 16743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:55.043635 16728 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260924 00:56:55.043819 16728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "3a538f6d12a64a48a45a88dc8773ab3f"
format_stamp: "Formatted at 2026-09-24 00:56:55 on dist-test-slave-mmhs"
server_key: "fe89985d82180ea1fba0b470a9cccc05"
server_key_iv: "bba8a335b756bc051e42ffb6babedd41"
server_key_version: "encryptionkey@0"
I20260924 00:56:55.044021 16728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:55.084142 16728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:55.088864 16728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:55.089174 16728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:55.108047 16728 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.190:42507
I20260924 00:56:55.108083 16794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.190:42507 every 8 connection(s)
I20260924 00:56:55.109452 16728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 00:56:55.113229 16795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:56:55.114010 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 16728
I20260924 00:56:55.114216 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 00:56:55.114876 8450 external_mini_cluster.cc:1428] Setting key d4a3b277a832248bd18a9e5a83e6e62f
I20260924 00:56:55.127282 16795 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a538f6d12a64a48a45a88dc8773ab3f: Bootstrap starting.
Sep 24 00:56:55 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211414, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:56:55.130614 16795 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a538f6d12a64a48a45a88dc8773ab3f: Neither blocks nor log segments found. Creating new log.
I20260924 00:56:55.132004 16795 log.cc:824] T 00000000000000000000000000000000 P 3a538f6d12a64a48a45a88dc8773ab3f: Log is configured to *not* fsync() on all Append() calls
I20260924 00:56:55.135206 16795 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a538f6d12a64a48a45a88dc8773ab3f: No bootstrap required, opened a new log
I20260924 00:56:55.138700 16798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:55.116949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59718 (local address 127.8.64.190:42507)
0924 00:56:55.117733 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:55.117750 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:55.117799 (+ 49us) server_negotiation.cc:407] Connection header received
0924 00:56:55.119700 (+ 1901us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:55.119728 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:55.120274 (+ 546us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:55.120720 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:55.121954 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:55.123978 (+ 2024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:55.125270 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:55.126282 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:55.126480 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:55.130469 (+ 3989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:55.130484 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:55.130509 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:55.130556 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:55.134327 (+ 3771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:55.134973 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:55.134981 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:55.134989 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:55.135174 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:55.135566 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:55.135573 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:55.135579 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:55.136289 (+ 710us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:55.136569 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:55.136866 (+ 297us) server_negotiation.cc:299] Negotiation successful
0924 00:56:55.137337 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":311,"threads_started":1}
I20260924 00:56:55.143898 16795 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a538f6d12a64a48a45a88dc8773ab3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a538f6d12a64a48a45a88dc8773ab3f" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42507 } }
I20260924 00:56:55.144207 16795 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3a538f6d12a64a48a45a88dc8773ab3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:56:55.144276 16795 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a538f6d12a64a48a45a88dc8773ab3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a538f6d12a64a48a45a88dc8773ab3f, State: Initialized, Role: FOLLOWER
I20260924 00:56:55.144784 16795 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a538f6d12a64a48a45a88dc8773ab3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a538f6d12a64a48a45a88dc8773ab3f" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42507 } }
I20260924 00:56:55.145006 16795 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3a538f6d12a64a48a45a88dc8773ab3f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:56:55.145112 16795 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a538f6d12a64a48a45a88dc8773ab3f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:56:55.145234 16795 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a538f6d12a64a48a45a88dc8773ab3f [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:56:55.147855 16795 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a538f6d12a64a48a45a88dc8773ab3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a538f6d12a64a48a45a88dc8773ab3f" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42507 } }
I20260924 00:56:55.148291 16795 leader_election.cc:302] T 00000000000000000000000000000000 P 3a538f6d12a64a48a45a88dc8773ab3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3a538f6d12a64a48a45a88dc8773ab3f; no voters:
I20260924 00:56:55.148921 16795 leader_election.cc:288] T 00000000000000000000000000000000 P 3a538f6d12a64a48a45a88dc8773ab3f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:56:55.149010 16800 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3a538f6d12a64a48a45a88dc8773ab3f [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:56:55.149744 16800 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3a538f6d12a64a48a45a88dc8773ab3f [term 1 LEADER]: Becoming Leader. State: Replica: 3a538f6d12a64a48a45a88dc8773ab3f, State: Running, Role: LEADER
I20260924 00:56:55.150350 16800 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a538f6d12a64a48a45a88dc8773ab3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a538f6d12a64a48a45a88dc8773ab3f" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42507 } }
I20260924 00:56:55.151074 16795 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a538f6d12a64a48a45a88dc8773ab3f [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:56:55.154861 16801 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a538f6d12a64a48a45a88dc8773ab3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3a538f6d12a64a48a45a88dc8773ab3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a538f6d12a64a48a45a88dc8773ab3f" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42507 } } }
I20260924 00:56:55.155494 16801 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a538f6d12a64a48a45a88dc8773ab3f [sys.catalog]: This master's current role is: LEADER
I20260924 00:56:55.157140 16802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a538f6d12a64a48a45a88dc8773ab3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a538f6d12a64a48a45a88dc8773ab3f. Latest consensus state: current_term: 1 leader_uuid: "3a538f6d12a64a48a45a88dc8773ab3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a538f6d12a64a48a45a88dc8773ab3f" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42507 } } }
I20260924 00:56:55.157600 16802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a538f6d12a64a48a45a88dc8773ab3f [sys.catalog]: This master's current role is: LEADER
I20260924 00:56:55.164487 16807 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:56:55.169961 16807 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:56:55.179711 16807 catalog_manager.cc:1347] Generated new cluster ID: b6998d77bc94434ebf2077fd5b34b2a8
I20260924 00:56:55.179833 16807 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:56:55.191097 16807 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:56:55.193152 16807 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:56:55.200635 16807 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3a538f6d12a64a48a45a88dc8773ab3f: Generated new TSK 0
I20260924 00:56:55.201690 16807 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.64.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab.
Entry for principal kudu/127.8.64.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab.
I20260924 00:56:55.273320 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.129:0
--local_ip_for_outbound_sockets=127.8.64.129
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:42507
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab
--principal=kudu/127.8.64.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:46155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:56:55.603492 16821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:55.603749 16821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:55.603844 16821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:55.610831 16821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:55.610927 16821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:55.611037 16821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.129
I20260924 00:56:55.620597 16821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:46155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab
--principal=kudu/127.8.64.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:42507
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.129
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:55.621836 16821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:55.623160 16821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:55.636108 16826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:55.636147 16827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:55.636071 16829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:55.636996 16821 server_base.cc:1034] running on GCE node
I20260924 00:56:55.638229 16821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:55.639201 16821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:55.640424 16821 hybrid_clock.cc:648] HybridClock initialized: now 1790211415640357 us; error 65 us; skew 500 ppm
I20260924 00:56:55.640726 16821 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:55 dist-test-slave-mmhs krb5kdc[16715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211415, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:55.643796 16821 init.cc:377] Logged in from keytab as kudu/127.8.64.129@KRBTEST.COM (short username kudu)
I20260924 00:56:55.645252 16821 webserver.cc:460] Webserver started at http://127.8.64.129:46725/ using document root <none> and password file <none>
I20260924 00:56:55.645941 16821 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:55.646085 16821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:55.646452 16821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:56:55.649375 16821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-0/data/instance:
uuid: "3807ed5708e9496b9b241d9c089c0dfa"
format_stamp: "Formatted at 2026-09-24 00:56:55 on dist-test-slave-mmhs"
server_key: "f24792a7666576fbd71e28aa53e045bc"
server_key_iv: "5667dc963e6b45f765021d9dc4ca6541"
server_key_version: "encryptionkey@0"
I20260924 00:56:55.650130 16821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance:
uuid: "3807ed5708e9496b9b241d9c089c0dfa"
format_stamp: "Formatted at 2026-09-24 00:56:55 on dist-test-slave-mmhs"
server_key: "f24792a7666576fbd71e28aa53e045bc"
server_key_iv: "5667dc963e6b45f765021d9dc4ca6541"
server_key_version: "encryptionkey@0"
I20260924 00:56:55.655356 16821 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260924 00:56:55.658807 16836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:55.660120 16821 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260924 00:56:55.660286 16821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-0/wal
uuid: "3807ed5708e9496b9b241d9c089c0dfa"
format_stamp: "Formatted at 2026-09-24 00:56:55 on dist-test-slave-mmhs"
server_key: "f24792a7666576fbd71e28aa53e045bc"
server_key_iv: "5667dc963e6b45f765021d9dc4ca6541"
server_key_version: "encryptionkey@0"
I20260924 00:56:55.660457 16821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:55.680989 16821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:55.685480 16821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:55.685783 16821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:55.686821 16821 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:56:55.688211 16821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:56:55.688284 16821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:55.688454 16821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:56:55.688514 16821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:55.721186 16821 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.129:46095
I20260924 00:56:55.721276 16949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.129:46095 every 8 connection(s)
I20260924 00:56:55.722566 16821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
I20260924 00:56:55.728971 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 16821
I20260924 00:56:55.729185 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance
I20260924 00:56:55.729815 8450 external_mini_cluster.cc:1428] Setting key d86db88d4c4f5cd1fd34028079ca6f96
Sep 24 00:56:55 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211415, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:56:55.742909 16952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:55.725061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:52353 (local address 127.8.64.190:42507)
0924 00:56:55.725509 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:55.725513 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:55.726385 (+ 872us) server_negotiation.cc:407] Connection header received
0924 00:56:55.728755 (+ 2370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:55.728761 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:55.728852 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:55.728978 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:55.730664 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:55.731655 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:55.733192 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:55.733576 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:55.733715 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:55.737058 (+ 3343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:55.737063 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:55.737067 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:55.737096 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:55.739690 (+ 2594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:55.740612 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:55.740615 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:55.740618 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:55.740734 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:55.741235 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:55.741238 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:55.741240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:55.741488 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:55.741595 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:55.742514 (+ 919us) server_negotiation.cc:299] Negotiation successful
0924 00:56:55.742669 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":248,"threads_started":1}
I20260924 00:56:55.744508 16953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:55.725537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.129:52353)
0924 00:56:55.726210 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:55.726252 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:55.728484 (+ 2232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:55.729224 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:55.729232 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:55.729775 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:55.730504 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:55.730530 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:55.731801 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:55.731807 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:55.733039 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:55.733050 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:55.733805 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:55.733812 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:55.734046 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:55.734738 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:55.734764 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:55.736835 (+ 2071us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:55.739836 (+ 3001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:55.739848 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:55.739868 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:55.740490 (+ 622us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:55.740841 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:55.740848 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:55.740855 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:55.741123 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:55.741610 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:55.741620 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:55.742081 (+ 461us) client_negotiation.cc:769] Sending connection context
0924 00:56:55.742420 (+ 339us) client_negotiation.cc:240] Negotiation successful
0924 00:56:55.743008 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":269,"threads_started":1}
I20260924 00:56:55.746199 16950 heartbeater.cc:347] Connected to a master server at 127.8.64.190:42507
I20260924 00:56:55.746551 16950 heartbeater.cc:464] Registering TS with master...
I20260924 00:56:55.747300 16950 heartbeater.cc:511] Master 127.8.64.190:42507 requested a full tablet report, sending...
I20260924 00:56:55.749954 16760 ts_manager.cc:194] Registered new tserver with Master: 3807ed5708e9496b9b241d9c089c0dfa (127.8.64.129:46095)
I20260924 00:56:55.752254 16760 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.129@KRBTEST.COM'} at 127.8.64.129:52353
WARNING: no policy specified for kudu/127.8.64.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab.
Entry for principal kudu/127.8.64.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab.
I20260924 00:56:55.782579 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.130:0
--local_ip_for_outbound_sockets=127.8.64.130
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:42507
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab
--principal=kudu/127.8.64.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:46155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:56:56.103242 16956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:56.103526 16956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:56.103618 16956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:56.112959 16956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:56.113090 16956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:56.113260 16956 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.130
I20260924 00:56:56.127606 16956 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:46155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab
--principal=kudu/127.8.64.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:42507
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.130
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:56.128846 16956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:56.130107 16956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:56.143246 16964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:56.143301 16961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:56.143335 16962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:56.144150 16956 server_base.cc:1034] running on GCE node
I20260924 00:56:56.144721 16956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:56.145601 16956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:56.146822 16956 hybrid_clock.cc:648] HybridClock initialized: now 1790211416146776 us; error 58 us; skew 500 ppm
I20260924 00:56:56.147131 16956 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:56 dist-test-slave-mmhs krb5kdc[16715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211416, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:56.150393 16956 init.cc:377] Logged in from keytab as kudu/127.8.64.130@KRBTEST.COM (short username kudu)
I20260924 00:56:56.151775 16956 webserver.cc:460] Webserver started at http://127.8.64.130:33585/ using document root <none> and password file <none>
I20260924 00:56:56.152491 16956 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:56.152590 16956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:56.152832 16956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:56:56.155625 16956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-1/data/instance:
uuid: "7d664d57013145f0a592fc4249dc45e9"
format_stamp: "Formatted at 2026-09-24 00:56:56 on dist-test-slave-mmhs"
server_key: "4116573530648c958d8d4ee7386e2fc5"
server_key_iv: "e297e478139268623ed1cb410ab52591"
server_key_version: "encryptionkey@0"
I20260924 00:56:56.156338 16956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance:
uuid: "7d664d57013145f0a592fc4249dc45e9"
format_stamp: "Formatted at 2026-09-24 00:56:56 on dist-test-slave-mmhs"
server_key: "4116573530648c958d8d4ee7386e2fc5"
server_key_iv: "e297e478139268623ed1cb410ab52591"
server_key_version: "encryptionkey@0"
I20260924 00:56:56.161294 16956 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260924 00:56:56.164726 16971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:56.166025 16956 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260924 00:56:56.166180 16956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-1/wal
uuid: "7d664d57013145f0a592fc4249dc45e9"
format_stamp: "Formatted at 2026-09-24 00:56:56 on dist-test-slave-mmhs"
server_key: "4116573530648c958d8d4ee7386e2fc5"
server_key_iv: "e297e478139268623ed1cb410ab52591"
server_key_version: "encryptionkey@0"
I20260924 00:56:56.166438 16956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:56.183281 16956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:56.187487 16956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:56.187773 16956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:56.188822 16956 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:56:56.190088 16956 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:56:56.190161 16956 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:56.190357 16956 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:56:56.190409 16956 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260924 00:56:56.222952 16956 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.130:41613
I20260924 00:56:56.222986 17083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.130:41613 every 8 connection(s)
I20260924 00:56:56.224198 16956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
I20260924 00:56:56.233971 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 16956
I20260924 00:56:56.234150 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance
I20260924 00:56:56.234724 8450 external_mini_cluster.cc:1428] Setting key 6b3c7d1f1a4ea6bfa7a764cd124405ef
Sep 24 00:56:56 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211416, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.130@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:56:56.244601 16952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:56.226685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:46615 (local address 127.8.64.190:42507)
0924 00:56:56.226899 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:56.226905 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:56.228052 (+ 1147us) server_negotiation.cc:407] Connection header received
0924 00:56:56.229743 (+ 1691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:56.229747 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:56.229823 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:56.229918 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:56.231430 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:56.232118 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:56.233643 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:56.234327 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:56.234488 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:56.237837 (+ 3349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:56.237847 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:56.237855 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:56.237898 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:56.241428 (+ 3530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:56.242126 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:56.242134 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:56.242140 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:56.242316 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:56.242713 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:56.242721 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:56.242727 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:56.243103 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:56.243259 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:56.243901 (+ 642us) server_negotiation.cc:299] Negotiation successful
0924 00:56:56.244150 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260924 00:56:56.245442 17086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:56.227197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.130:46615)
0924 00:56:56.227885 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:56.227939 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:56.229491 (+ 1552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:56.230135 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:56.230142 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:56.230714 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:56.231282 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:56.231300 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:56.232258 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:56.232262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:56.233495 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:56.233506 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:56.234564 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:56.234569 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:56.234756 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:56.235487 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:56.235512 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:56.237607 (+ 2095us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:56.241565 (+ 3958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:56.241572 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:56.241584 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:56.242007 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:56.242435 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:56.242438 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:56.242440 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:56.242606 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:56.243227 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:56.243233 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:56.243549 (+ 316us) client_negotiation.cc:769] Sending connection context
0924 00:56:56.243776 (+ 227us) client_negotiation.cc:240] Negotiation successful
0924 00:56:56.244149 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":285,"threads_started":1}
I20260924 00:56:56.246930 17084 heartbeater.cc:347] Connected to a master server at 127.8.64.190:42507
I20260924 00:56:56.247234 17084 heartbeater.cc:464] Registering TS with master...
I20260924 00:56:56.248019 17084 heartbeater.cc:511] Master 127.8.64.190:42507 requested a full tablet report, sending...
I20260924 00:56:56.249505 16760 ts_manager.cc:194] Registered new tserver with Master: 7d664d57013145f0a592fc4249dc45e9 (127.8.64.130:41613)
I20260924 00:56:56.250332 16760 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.130@KRBTEST.COM'} at 127.8.64.130:46615
WARNING: no policy specified for kudu/127.8.64.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab.
Entry for principal kudu/127.8.64.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab.
I20260924 00:56:56.285632 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.131:0
--local_ip_for_outbound_sockets=127.8.64.131
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:42507
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab
--principal=kudu/127.8.64.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:46155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:56:56.607208 17089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:56.607465 17089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:56.607651 17089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:56.614691 17089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:56.614806 17089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:56.614986 17089 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.131
I20260924 00:56:56.624384 17089 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:46155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab
--principal=kudu/127.8.64.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:42507
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.131
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:56.625607 17089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:56.626978 17089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:56.639853 17097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:56.639896 17095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:56.640087 17094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:56.640870 17089 server_base.cc:1034] running on GCE node
I20260924 00:56:56.641458 17089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:56.642546 17089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:56.643767 17089 hybrid_clock.cc:648] HybridClock initialized: now 1790211416643730 us; error 53 us; skew 500 ppm
I20260924 00:56:56.644083 17089 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:56 dist-test-slave-mmhs krb5kdc[16715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211416, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:56.647300 17089 init.cc:377] Logged in from keytab as kudu/127.8.64.131@KRBTEST.COM (short username kudu)
I20260924 00:56:56.648838 17089 webserver.cc:460] Webserver started at http://127.8.64.131:42255/ using document root <none> and password file <none>
I20260924 00:56:56.649462 17089 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:56.649559 17089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:56.649802 17089 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:56:56.652726 17089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-2/data/instance:
uuid: "b68e04bac8b1447a8b5745d29d56c6d8"
format_stamp: "Formatted at 2026-09-24 00:56:56 on dist-test-slave-mmhs"
server_key: "2524c8fd47c75818324cb5dd39f10137"
server_key_iv: "87ceb3320ab758eef6b174c13737a4ff"
server_key_version: "encryptionkey@0"
I20260924 00:56:56.653442 17089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance:
uuid: "b68e04bac8b1447a8b5745d29d56c6d8"
format_stamp: "Formatted at 2026-09-24 00:56:56 on dist-test-slave-mmhs"
server_key: "2524c8fd47c75818324cb5dd39f10137"
server_key_iv: "87ceb3320ab758eef6b174c13737a4ff"
server_key_version: "encryptionkey@0"
I20260924 00:56:56.658383 17089 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260924 00:56:56.661639 17104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:56.663071 17089 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 00:56:56.663223 17089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-2/wal
uuid: "b68e04bac8b1447a8b5745d29d56c6d8"
format_stamp: "Formatted at 2026-09-24 00:56:56 on dist-test-slave-mmhs"
server_key: "2524c8fd47c75818324cb5dd39f10137"
server_key_iv: "87ceb3320ab758eef6b174c13737a4ff"
server_key_version: "encryptionkey@0"
I20260924 00:56:56.663422 17089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:56.677903 17089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:56.682286 17089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:56.682679 17089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:56.683662 17089 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:56:56.684916 17089 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:56:56.684985 17089 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:56.685156 17089 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:56:56.685230 17089 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:56.718029 17089 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.131:43379
I20260924 00:56:56.718076 17217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.131:43379 every 8 connection(s)
I20260924 00:56:56.719290 17089 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
I20260924 00:56:56.726080 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 17089
I20260924 00:56:56.726243 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance
I20260924 00:56:56.726837 8450 external_mini_cluster.cc:1428] Setting key 0f0ee2d76ded723218669ff713db2b1d
Sep 24 00:56:56 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211416, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.131@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:56:56.739454 16952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:56.722247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:47369 (local address 127.8.64.190:42507)
0924 00:56:56.722475 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:56.722481 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:56.723661 (+ 1180us) server_negotiation.cc:407] Connection header received
0924 00:56:56.725477 (+ 1816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:56.725485 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:56.725613 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:56.725761 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:56.727721 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:56.728631 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:56.730567 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:56.730982 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:56.731075 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:56.734375 (+ 3300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:56.734381 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:56.734384 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:56.734414 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:56.736797 (+ 2383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:56.737426 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:56.737430 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:56.737432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:56.737606 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:56.737990 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:56.737993 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:56.737995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:56.738253 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:56.738408 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:56.739053 (+ 645us) server_negotiation.cc:299] Negotiation successful
0924 00:56:56.739212 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260924 00:56:56.740351 17220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:56.722665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.131:47369)
0924 00:56:56.723476 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:56.723522 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:56.725179 (+ 1657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:56.726115 (+ 936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:56.726126 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:56.726859 (+ 733us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:56.727570 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:56.727587 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:56.728776 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:56.728780 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:56.730436 (+ 1656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:56.730447 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:56.731154 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:56.731158 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:56.731348 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:56.732011 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:56.732031 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:56.734112 (+ 2081us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:56:56.736928 (+ 2816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:56.736934 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:56.736945 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:56.737312 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:56.737714 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:56:56.737717 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:56:56.737719 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:56:56.737878 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:56:56.738425 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:56:56.738431 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:56:56.738706 (+ 275us) client_negotiation.cc:769] Sending connection context
0924 00:56:56.738923 (+ 217us) client_negotiation.cc:240] Negotiation successful
0924 00:56:56.739266 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":346,"threads_started":1}
I20260924 00:56:56.741781 17218 heartbeater.cc:347] Connected to a master server at 127.8.64.190:42507
I20260924 00:56:56.742070 17218 heartbeater.cc:464] Registering TS with master...
I20260924 00:56:56.742831 17218 heartbeater.cc:511] Master 127.8.64.190:42507 requested a full tablet report, sending...
I20260924 00:56:56.744145 16760 ts_manager.cc:194] Registered new tserver with Master: b68e04bac8b1447a8b5745d29d56c6d8 (127.8.64.131:43379)
I20260924 00:56:56.744943 16760 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.131@KRBTEST.COM'} at 127.8.64.131:47369
I20260924 00:56:56.754283 8450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 00:56:56.755355 16950 heartbeater.cc:503] Master 127.8.64.190:42507 was elected leader, sending a full tablet report...
WARNING: no policy specified for oryx/127.8.64.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.64.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.64.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.129.keytab.
Entry for principal oryx/127.8.64.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.129.keytab.
WARNING: no policy specified for oryx/127.8.64.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.64.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.64.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.130.keytab.
Entry for principal oryx/127.8.64.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.130.keytab.
WARNING: no policy specified for oryx/127.8.64.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.64.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.64.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.131.keytab.
Entry for principal oryx/127.8.64.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.131.keytab.
I20260924 00:56:56.899416 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 16821
Sep 24 00:56:56 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211414, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.129@KRBTEST.COM
I20260924 00:56:56.915518 17228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:56.900496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35052 (local address 127.8.64.129:46095)
0924 00:56:56.901107 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:56.901114 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:56.901142 (+ 28us) server_negotiation.cc:407] Connection header received
0924 00:56:56.901203 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:56.901209 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:56.901435 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:56.901575 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:56.902864 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:56.903862 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:56.905146 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:56.905965 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:56.906090 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:56.909855 (+ 3765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:56.909868 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:56.909884 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:56.909915 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:56.912866 (+ 2951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:56.913571 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:56.913579 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:56.913581 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:56.913710 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:56.914128 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:56.914132 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:56.914134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:56.914554 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:56.914774 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:56.915022 (+ 248us) server_negotiation.cc:299] Negotiation successful
0924 00:56:56.915213 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":314,"threads_started":1}
I20260924 00:56:56.916772 16945 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:56:56.998198 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 16821
I20260924 00:56:57.015013 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 16956
Sep 24 00:56:57 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211414, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.130@KRBTEST.COM
I20260924 00:56:57.031210 17230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:57.015878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47036 (local address 127.8.64.130:41613)
0924 00:56:57.016423 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:57.016431 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:57.016455 (+ 24us) server_negotiation.cc:407] Connection header received
0924 00:56:57.016518 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:57.016525 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:57.016721 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:57.016905 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:57.018125 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:57.019157 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:57.020435 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:57.020966 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:57.021100 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:57.025807 (+ 4707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:57.025817 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:57.025831 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:57.025859 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:57.028486 (+ 2627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:57.029119 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:57.029126 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:57.029128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:57.029252 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:57.029623 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:57.029629 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:57.029632 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:57.029962 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:57.030256 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:57.030595 (+ 339us) server_negotiation.cc:299] Negotiation successful
0924 00:56:57.030857 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":303,"threads_started":1}
I20260924 00:56:57.032444 17079 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:56:57.114878 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 16956
I20260924 00:56:57.131161 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 17089
Sep 24 00:56:57 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211414, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.131@KRBTEST.COM
I20260924 00:56:57.146675 17232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:57.132149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48062 (local address 127.8.64.131:43379)
0924 00:56:57.132723 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:57.132730 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:57.132758 (+ 28us) server_negotiation.cc:407] Connection header received
0924 00:56:57.132813 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:57.132819 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:57.133025 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:57.133148 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:57.134418 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:57.135464 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:57.136724 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:57.137280 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:57.137392 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:57.141537 (+ 4145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:57.141547 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:57.141561 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:57.141593 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:57.144394 (+ 2801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:57.144958 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:57.144964 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:57.144966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:57.145082 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:57.145379 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:57.145382 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:57.145384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:57.145692 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:57.145926 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:57.146126 (+ 200us) server_negotiation.cc:299] Negotiation successful
0924 00:56:57.146363 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":289,"threads_started":1}
I20260924 00:56:57.147902 17213 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:56:57.232049 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 17089
I20260924 00:56:57.247925 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 16728
I20260924 00:56:57.248993 16790 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:56:57.349514 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 16728
I20260924 00:56:57.369805 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:42507
--webserver_interface=127.8.64.190
--webserver_port=44245
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:46155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:42507
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:56:57.687233 17235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:57.687518 17235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:57.687698 17235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:57.694710 17235 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:56:57.694828 17235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:57.694908 17235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:57.695001 17235 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:56:57.695075 17235 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:56:57.704463 17235 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:46155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:42507
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:42507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=44245
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:57.705699 17235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:57.706995 17235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:57.718976 17241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:57.718978 17243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:57.719026 17240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:57.720198 17235 server_base.cc:1034] running on GCE node
I20260924 00:56:57.721122 17235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:57.722755 17235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:57.724020 17235 hybrid_clock.cc:648] HybridClock initialized: now 1790211417723954 us; error 62 us; skew 500 ppm
I20260924 00:56:57.724370 17235 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:57 dist-test-slave-mmhs krb5kdc[16715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211417, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:57.727471 17235 init.cc:377] Logged in from keytab as kudu/127.8.64.190@KRBTEST.COM (short username kudu)
I20260924 00:56:57.728878 17235 webserver.cc:460] Webserver started at http://127.8.64.190:44245/ using document root <none> and password file <none>
I20260924 00:56:57.729635 17235 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:57.729789 17235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:57.736692 17235 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260924 00:56:57.740603 17250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:57.741931 17235 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260924 00:56:57.742118 17235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "3a538f6d12a64a48a45a88dc8773ab3f"
format_stamp: "Formatted at 2026-09-24 00:56:55 on dist-test-slave-mmhs"
server_key: "fe89985d82180ea1fba0b470a9cccc05"
server_key_iv: "bba8a335b756bc051e42ffb6babedd41"
server_key_version: "encryptionkey@0"
I20260924 00:56:57.743124 17235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:57.771529 17235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:57.775843 17235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:57.776144 17235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:57.794903 17235 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.190:42507
I20260924 00:56:57.794924 17301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.190:42507 every 8 connection(s)
I20260924 00:56:57.796425 17235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 00:56:57.800609 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 17235
I20260924 00:56:57.801429 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.129:46095
--local_ip_for_outbound_sockets=127.8.64.129
--tserver_master_addrs=127.8.64.190:42507
--webserver_port=46725
--webserver_interface=127.8.64.129
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab
--principal=kudu/127.8.64.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:46155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.64.129
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 00:56:57.803363 17302 sys_catalog.cc:263] Verifying existing consensus state
I20260924 00:56:57.806536 17302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a538f6d12a64a48a45a88dc8773ab3f: Bootstrap starting.
I20260924 00:56:57.832935 17302 log.cc:824] T 00000000000000000000000000000000 P 3a538f6d12a64a48a45a88dc8773ab3f: Log is configured to *not* fsync() on all Append() calls
I20260924 00:56:57.843953 17302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a538f6d12a64a48a45a88dc8773ab3f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 00:56:57.844568 17302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a538f6d12a64a48a45a88dc8773ab3f: Bootstrap complete.
I20260924 00:56:57.857785 17302 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a538f6d12a64a48a45a88dc8773ab3f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a538f6d12a64a48a45a88dc8773ab3f" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42507 } }
I20260924 00:56:57.858960 17302 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a538f6d12a64a48a45a88dc8773ab3f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a538f6d12a64a48a45a88dc8773ab3f, State: Initialized, Role: FOLLOWER
I20260924 00:56:57.859664 17302 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a538f6d12a64a48a45a88dc8773ab3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a538f6d12a64a48a45a88dc8773ab3f" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42507 } }
I20260924 00:56:57.859992 17302 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3a538f6d12a64a48a45a88dc8773ab3f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:56:57.860134 17302 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a538f6d12a64a48a45a88dc8773ab3f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:56:57.860327 17302 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a538f6d12a64a48a45a88dc8773ab3f [term 1 FOLLOWER]: Advancing to term 2
I20260924 00:56:57.865954 17302 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a538f6d12a64a48a45a88dc8773ab3f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a538f6d12a64a48a45a88dc8773ab3f" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42507 } }
I20260924 00:56:57.866657 17302 leader_election.cc:302] T 00000000000000000000000000000000 P 3a538f6d12a64a48a45a88dc8773ab3f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3a538f6d12a64a48a45a88dc8773ab3f; no voters:
I20260924 00:56:57.867385 17302 leader_election.cc:288] T 00000000000000000000000000000000 P 3a538f6d12a64a48a45a88dc8773ab3f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260924 00:56:57.867530 17306 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3a538f6d12a64a48a45a88dc8773ab3f [term 2 FOLLOWER]: Leader election won for term 2
I20260924 00:56:57.868904 17306 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3a538f6d12a64a48a45a88dc8773ab3f [term 2 LEADER]: Becoming Leader. State: Replica: 3a538f6d12a64a48a45a88dc8773ab3f, State: Running, Role: LEADER
I20260924 00:56:57.869418 17306 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a538f6d12a64a48a45a88dc8773ab3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a538f6d12a64a48a45a88dc8773ab3f" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42507 } }
I20260924 00:56:57.869824 17302 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a538f6d12a64a48a45a88dc8773ab3f [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:56:57.871824 17308 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a538f6d12a64a48a45a88dc8773ab3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a538f6d12a64a48a45a88dc8773ab3f. Latest consensus state: current_term: 2 leader_uuid: "3a538f6d12a64a48a45a88dc8773ab3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a538f6d12a64a48a45a88dc8773ab3f" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42507 } } }
I20260924 00:56:57.872255 17308 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a538f6d12a64a48a45a88dc8773ab3f [sys.catalog]: This master's current role is: LEADER
I20260924 00:56:57.872815 17307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a538f6d12a64a48a45a88dc8773ab3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "3a538f6d12a64a48a45a88dc8773ab3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a538f6d12a64a48a45a88dc8773ab3f" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42507 } } }
I20260924 00:56:57.873160 17307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a538f6d12a64a48a45a88dc8773ab3f [sys.catalog]: This master's current role is: LEADER
I20260924 00:56:57.876732 17312 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:56:57.882670 17312 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:56:57.884054 17312 catalog_manager.cc:1259] Loaded cluster ID: b6998d77bc94434ebf2077fd5b34b2a8
I20260924 00:56:57.884200 17312 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:56:57.888425 17312 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:56:57.889866 17312 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3a538f6d12a64a48a45a88dc8773ab3f: Loaded TSK: 0
I20260924 00:56:57.891518 17312 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260924 00:56:58.151607 17304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:58.151865 17304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:58.152000 17304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:58.159171 17304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:58.159279 17304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:58.159448 17304 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.129
I20260924 00:56:58.169004 17304 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:46155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.129.keytab
--principal=oryx/127.8.64.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.129:46095
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.64.129
--webserver_port=46725
--tserver_master_addrs=127.8.64.190:42507
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.129
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:58.170483 17304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:58.171866 17304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:58.184628 17329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:58.184664 17330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:58.185109 17332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:58.186369 17304 server_base.cc:1034] running on GCE node
I20260924 00:56:58.186936 17304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:58.187822 17304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:58.189059 17304 hybrid_clock.cc:648] HybridClock initialized: now 1790211418189007 us; error 48 us; skew 500 ppm
I20260924 00:56:58.189363 17304 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:58 dist-test-slave-mmhs krb5kdc[16715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211418, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.64.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:58.192590 17304 init.cc:377] Logged in from keytab as oryx/127.8.64.129@KRBTEST.COM (short username oryx)
I20260924 00:56:58.194585 17304 webserver.cc:460] Webserver started at http://127.8.64.129:46725/ using document root <none> and password file <none>
I20260924 00:56:58.195269 17304 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:58.195423 17304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:58.200682 17304 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.005s
I20260924 00:56:58.203382 17339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:58.204735 17304 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260924 00:56:58.204921 17304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-0/wal
uuid: "3807ed5708e9496b9b241d9c089c0dfa"
format_stamp: "Formatted at 2026-09-24 00:56:55 on dist-test-slave-mmhs"
server_key: "f24792a7666576fbd71e28aa53e045bc"
server_key_iv: "5667dc963e6b45f765021d9dc4ca6541"
server_key_version: "encryptionkey@0"
I20260924 00:56:58.205837 17304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:58.219553 17304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:58.223603 17304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:58.223891 17304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:58.224814 17304 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:56:58.226200 17304 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:56:58.226354 17304 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:58.226477 17304 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:56:58.226567 17304 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:58.259265 17304 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.129:46095
I20260924 00:56:58.259349 17452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.129:46095 every 8 connection(s)
I20260924 00:56:58.260689 17304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
I20260924 00:56:58.267439 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 17304
I20260924 00:56:58.268119 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.130:41613
--local_ip_for_outbound_sockets=127.8.64.130
--tserver_master_addrs=127.8.64.190:42507
--webserver_port=33585
--webserver_interface=127.8.64.130
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab
--principal=kudu/127.8.64.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:46155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.64.130
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 00:56:58 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.129@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:56:58 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.129@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:56:58.281770 17456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:58.263470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.129:41025)
0924 00:56:58.264459 (+ 989us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:58.264528 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:58.267486 (+ 2958us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:58.268931 (+ 1445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:58.268945 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:58.269755 (+ 810us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:58.270583 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:58.270609 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:58.272308 (+ 1699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:58.272316 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:58.274469 (+ 2153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:58.274491 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:58.275485 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:58.275493 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:58.275769 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:58.276486 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:58.276507 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:58.280452 (+ 3945us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":350,"threads_started":1}
W20260924 00:56:58.281797 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:56:58.263514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:41025 (local address 127.8.64.190:42507)
0924 00:56:58.266725 (+ 3211us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:58.266737 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:58.266779 (+ 42us) server_negotiation.cc:407] Connection header received
0924 00:56:58.268014 (+ 1235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:58.268027 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:58.268441 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:58.268726 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:58.270786 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:58.272137 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:58.274610 (+ 2473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:58.275247 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:58.275470 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:58.280807 (+ 5337us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.129:41025: BlockingRecv error: recv got EOF from 127.8.64.129:41025 (error 108)
Metrics: {"server-negotiator.queue_time_us":3018,"thread_start_us":2870,"threads_started":1}
W20260924 00:56:58.282773 17453 heartbeater.cc:644] Failed to heartbeat to 127.8.64.190:42507 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.64.190:42507: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Sep 24 00:56:58 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.129@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:56:58 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.129@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:56:58.293012 17456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:58.283469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.129:58497)
0924 00:56:58.283683 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:58.283706 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:58.283842 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:58.284174 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:58.284182 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:58.284633 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:58.284851 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:58.284865 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:58.286038 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:58.286046 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:58.287880 (+ 1834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:58.287897 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:58.288649 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:58.288657 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:58.288900 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:58.289548 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:58.289577 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:58.292807 (+ 3230us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":59}
W20260924 00:56:58.293655 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:56:58.283548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:58497 (local address 127.8.64.190:42507)
0924 00:56:58.283706 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:58.283710 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:58.283784 (+ 74us) server_negotiation.cc:407] Connection header received
0924 00:56:58.283969 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:58.283973 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:58.284028 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:58.284108 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:58.285086 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:58.285869 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:58.288061 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:58.288488 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:58.288586 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:58.293102 (+ 4516us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.129:58497: BlockingRecv error: recv got EOF from 127.8.64.129:58497 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 24 00:56:58 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.129@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:56:58 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.129@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:56:58.302045 17456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:58.293710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.129:48691)
0924 00:56:58.293947 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:58.293967 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:58.294058 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:58.294455 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:58.294459 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:58.294820 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:58.295005 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:58.295015 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:58.295969 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:58.295974 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:58.297445 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:58.297459 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:58.298086 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:58.298094 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:58.298287 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:58.298850 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:58.298872 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:58.301844 (+ 2972us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107}
W20260924 00:56:58.302361 17453 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 00:56:58.302661 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:56:58.293939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:48691 (local address 127.8.64.190:42507)
0924 00:56:58.294106 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:58.294111 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:58.294127 (+ 16us) server_negotiation.cc:407] Connection header received
0924 00:56:58.294181 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:58.294185 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:58.294257 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:58.294409 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:58.295134 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:58.295820 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:58.297580 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:58.297960 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:58.298045 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:58.302086 (+ 4041us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.129:48691: BlockingRecv error: recv got EOF from 127.8.64.129:48691 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260924 00:56:58.597086 17457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:58.597348 17457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:58.597448 17457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:58.604475 17457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:58.604578 17457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:58.604697 17457 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.130
I20260924 00:56:58.614145 17457 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:46155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.130.keytab
--principal=oryx/127.8.64.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.130:41613
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.64.130
--webserver_port=33585
--tserver_master_addrs=127.8.64.190:42507
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.130
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:58.615343 17457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:58.616569 17457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:58.629573 17463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:58.629622 17465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:58.629658 17462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:58.630162 17457 server_base.cc:1034] running on GCE node
I20260924 00:56:58.630843 17457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:58.631806 17457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:58.633029 17457 hybrid_clock.cc:648] HybridClock initialized: now 1790211418632982 us; error 59 us; skew 500 ppm
I20260924 00:56:58.633335 17457 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:58 dist-test-slave-mmhs krb5kdc[16715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211418, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.64.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:58.636355 17457 init.cc:377] Logged in from keytab as oryx/127.8.64.130@KRBTEST.COM (short username oryx)
I20260924 00:56:58.637705 17457 webserver.cc:460] Webserver started at http://127.8.64.130:33585/ using document root <none> and password file <none>
I20260924 00:56:58.638425 17457 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:58.638578 17457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:58.643721 17457 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260924 00:56:58.646394 17472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:58.647748 17457 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 00:56:58.647918 17457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-1/wal
uuid: "7d664d57013145f0a592fc4249dc45e9"
format_stamp: "Formatted at 2026-09-24 00:56:56 on dist-test-slave-mmhs"
server_key: "4116573530648c958d8d4ee7386e2fc5"
server_key_iv: "e297e478139268623ed1cb410ab52591"
server_key_version: "encryptionkey@0"
I20260924 00:56:58.648842 17457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:58.673017 17457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:58.677338 17457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:58.677654 17457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:58.678753 17457 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:56:58.680109 17457 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:56:58.680234 17457 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:58.680414 17457 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:56:58.680501 17457 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:58.713107 17457 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.130:41613
I20260924 00:56:58.713196 17584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.130:41613 every 8 connection(s)
I20260924 00:56:58.714680 17457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
I20260924 00:56:58.722992 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 17457
I20260924 00:56:58.723698 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.131:43379
--local_ip_for_outbound_sockets=127.8.64.131
--tserver_master_addrs=127.8.64.190:42507
--webserver_port=42255
--webserver_interface=127.8.64.131
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab
--principal=kudu/127.8.64.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:46155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.64.131
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 00:56:58 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.130@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:56:58 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.130@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
W20260924 00:56:58.731541 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:56:58.717197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:32913 (local address 127.8.64.190:42507)
0924 00:56:58.717480 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:58.717485 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:58.718525 (+ 1040us) server_negotiation.cc:407] Connection header received
0924 00:56:58.720241 (+ 1716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:58.720245 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:58.720316 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:58.720402 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:58.721810 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:58.722678 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:58.725127 (+ 2449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:58.725558 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:58.725653 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:58.731268 (+ 5615us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.130:32913: BlockingRecv error: recv got EOF from 127.8.64.130:32913 (error 108)
Metrics: {"server-negotiator.queue_time_us":159}
I20260924 00:56:58.732250 17587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:58.717631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.130:32913)
0924 00:56:58.718362 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:58.718398 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:58.719991 (+ 1593us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:58.720608 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:58.720615 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:58.721086 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:58.721631 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:58.721648 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:58.722841 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:58.722846 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:58.724980 (+ 2134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:58.724999 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:58.725786 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:58.725795 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:58.726110 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:58.726831 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:58.726864 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:58.731099 (+ 4235us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":275,"threads_started":1}
W20260924 00:56:58.732913 17585 heartbeater.cc:644] Failed to heartbeat to 127.8.64.190:42507 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.64.190:42507: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Sep 24 00:56:58 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.130@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:56:58 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.130@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:56:58.741488 17587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:58.733507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.130:55381)
0924 00:56:58.733690 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:58.733717 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:58.733797 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:58.734121 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:58.734124 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:58.734484 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:58.734685 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:58.734697 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:58.735999 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:58.736006 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:58.736885 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:58.736896 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:58.737565 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:58.737570 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:58.737738 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:58.738368 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:58.738392 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:58.741167 (+ 2775us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":56}
W20260924 00:56:58.741611 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:56:58.733612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:55381 (local address 127.8.64.190:42507)
0924 00:56:58.733810 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:58.733818 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:58.733839 (+ 21us) server_negotiation.cc:407] Connection header received
0924 00:56:58.733896 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:58.733904 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:58.733994 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:58.734097 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:58.734850 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:58.735847 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:58.737022 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:58.737425 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:58.737554 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:58.741366 (+ 3812us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.130:55381: BlockingRecv error: recv got EOF from 127.8.64.130:55381 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 24 00:56:58 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.130@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:56:58 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.130@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:56:58.749692 17587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:58.742127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.130:52473)
0924 00:56:58.742403 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:58.742421 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:58.742518 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:58.742808 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:58.742812 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:58.743101 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:58.743234 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:58.743242 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:58.744199 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:58.744204 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:58.745093 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:58.745103 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:58.745798 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:58.745803 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:58.745977 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:58.746578 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:58.746623 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:58.749394 (+ 2771us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122}
W20260924 00:56:58.750012 17585 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 00:56:58.750195 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:56:58.742328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:52473 (local address 127.8.64.190:42507)
0924 00:56:58.742510 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:58.742515 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:58.742533 (+ 18us) server_negotiation.cc:407] Connection header received
0924 00:56:58.742616 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:58.742621 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:58.742689 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:58.742787 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:58.743367 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:58.744070 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:58.745233 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:58.745654 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:58.745789 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:58.749607 (+ 3818us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.130:52473: BlockingRecv error: recv got EOF from 127.8.64.130:52473 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260924 00:56:59.049739 17588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:56:59.050055 17588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:56:59.050225 17588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:56:59.057839 17588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:56:59.057986 17588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:56:59.058151 17588 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.131
I20260924 00:56:59.067826 17588 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:46155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/krb5kdc/oryx_127.8.64.131.keytab
--principal=oryx/127.8.64.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.131:43379
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.64.131
--webserver_port=42255
--tserver_master_addrs=127.8.64.190:42507
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.131
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:56:59.069151 17588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:56:59.070607 17588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:56:59.083663 17593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:59.083731 17594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:56:59.083673 17596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:56:59.084569 17588 server_base.cc:1034] running on GCE node
I20260924 00:56:59.085198 17588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:56:59.086153 17588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:56:59.087456 17588 hybrid_clock.cc:648] HybridClock initialized: now 1790211419087356 us; error 66 us; skew 500 ppm
I20260924 00:56:59.087883 17588 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:56:59 dist-test-slave-mmhs krb5kdc[16715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211419, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.64.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:56:59.091008 17588 init.cc:377] Logged in from keytab as oryx/127.8.64.131@KRBTEST.COM (short username oryx)
I20260924 00:56:59.092411 17588 webserver.cc:460] Webserver started at http://127.8.64.131:42255/ using document root <none> and password file <none>
I20260924 00:56:59.093096 17588 fs_manager.cc:359] Metadata directory not provided
I20260924 00:56:59.093248 17588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:56:59.098449 17588 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260924 00:56:59.100935 17604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:59.102227 17588 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260924 00:56:59.102442 17588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-2/wal
uuid: "b68e04bac8b1447a8b5745d29d56c6d8"
format_stamp: "Formatted at 2026-09-24 00:56:56 on dist-test-slave-mmhs"
server_key: "2524c8fd47c75818324cb5dd39f10137"
server_key_iv: "87ceb3320ab758eef6b174c13737a4ff"
server_key_version: "encryptionkey@0"
I20260924 00:56:59.103355 17588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:56:59.129312 17588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:56:59.133613 17588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:56:59.133890 17588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:56:59.135020 17588 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:56:59.136335 17588 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:56:59.136456 17588 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:59.136587 17588 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:56:59.136674 17588 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:56:59.169103 17588 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.131:43379
I20260924 00:56:59.169188 17716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.131:43379 every 8 connection(s)
I20260924 00:56:59.170763 17588 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
I20260924 00:56:59.179028 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 17588
Sep 24 00:56:59 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.131@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:56:59 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.131@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
W20260924 00:56:59.187536 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:56:59.173294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:58853 (local address 127.8.64.190:42507)
0924 00:56:59.173500 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:59.173506 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:59.174780 (+ 1274us) server_negotiation.cc:407] Connection header received
0924 00:56:59.176549 (+ 1769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:59.176553 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:59.176618 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:59.176706 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:59.178243 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:59.179191 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:59.181444 (+ 2253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:59.181858 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:59.182020 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:59.187282 (+ 5262us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.131:58853: BlockingRecv error: recv got EOF from 127.8.64.131:58853 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260924 00:56:59.188413 17719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:59.173871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.131:58853)
0924 00:56:59.174624 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:59.174670 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:59.176293 (+ 1623us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:59.176908 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:59.176916 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:59.177407 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:59.177981 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:59.177999 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:59.179355 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:59.179361 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:59.181297 (+ 1936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:59.181309 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:59.182096 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:59.182103 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:59.182395 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:59.183108 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:59.183139 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:59.187125 (+ 3986us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":269,"threads_started":1}
W20260924 00:56:59.189411 17717 heartbeater.cc:644] Failed to heartbeat to 127.8.64.190:42507 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.64.190:42507: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Sep 24 00:56:59 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.131@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:56:59 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.131@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:56:59.199175 17719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:59.190098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.131:43661)
0924 00:56:59.190371 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:59.190390 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:59.190510 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:59.190859 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:59.190888 (+ 29us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:59.191190 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:59.191315 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:59.191323 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:59.192240 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:59.192246 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:59.193502 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:59.193519 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:59.194477 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:59.194484 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:59.194668 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:59.195334 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:59.195358 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:59.198894 (+ 3536us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143}
W20260924 00:56:59.199311 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:56:59.190231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:43661 (local address 127.8.64.190:42507)
0924 00:56:59.190472 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:59.190478 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:59.190522 (+ 44us) server_negotiation.cc:407] Connection header received
0924 00:56:59.190629 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:59.190634 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:59.190721 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:59.190809 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:59.191453 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:59.192111 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:59.193666 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:59.194263 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:59.194435 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:59.199074 (+ 4639us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.131:43661: BlockingRecv error: recv got EOF from 127.8.64.131:43661 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260924 00:56:59.201081 17721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:59.180104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59728 (local address 127.8.64.190:42507)
0924 00:56:59.181130 (+ 1026us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:59.181134 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:59.181147 (+ 13us) server_negotiation.cc:407] Connection header received
0924 00:56:59.181183 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:59.181186 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:59.181235 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:59.181361 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:59.182861 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:59.184022 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:59.185312 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:59.185855 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:59.185999 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:59.192892 (+ 6893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:56:59.192902 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:56:59.192922 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:56:59.192982 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:56:59.197389 (+ 4407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:59.198240 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:59.198248 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:59.198257 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:59.198479 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:56:59.198844 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:56:59.198851 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:56:59.198854 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:56:59.199393 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:56:59.199668 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0924 00:56:59.200425 (+ 757us) server_negotiation.cc:299] Negotiation successful
0924 00:56:59.200781 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":944,"thread_start_us":851,"threads_started":1}
Sep 24 00:56:59 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.131@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:56:59 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.131@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:56:59.207899 17719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:59.199834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.131:55299)
0924 00:56:59.200088 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:59.200106 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:59.200199 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:59.200489 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:59.200493 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:59.200787 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:59.200913 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:59.200920 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:59.202046 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:59.202052 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:59.203377 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:59.203389 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:59.204152 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:59.204157 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:59.204339 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:59.204911 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:59.204926 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:59.207595 (+ 2669us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145}
W20260924 00:56:59.207913 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:56:59.199966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:55299 (local address 127.8.64.190:42507)
0924 00:56:59.200181 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:59.200187 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:59.200204 (+ 17us) server_negotiation.cc:407] Connection header received
0924 00:56:59.200295 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:59.200300 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:59.200368 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:59.200473 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:59.201058 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:59.201914 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:59.203535 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:59.204010 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:59.204111 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:59.207754 (+ 3643us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.131:55299: BlockingRecv error: recv got EOF from 127.8.64.131:55299 (error 108)
Metrics: {"server-negotiator.queue_time_us":119}
W20260924 00:56:59.208192 17717 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 24 00:56:59 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.129@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:56:59 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.129@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:56:59.311724 17722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:59.302886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.129:41967)
0924 00:56:59.303388 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:59.303404 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:59.303550 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:59.303876 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:59.303881 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:59.304364 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:59.304572 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:59.304580 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:59.305570 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:59.305576 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:59.306576 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:59.306587 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:59.307311 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:59.307317 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:59.307497 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:59.308102 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:59.308119 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:59.311432 (+ 3313us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":250,"threads_started":1}
W20260924 00:56:59.311815 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:56:59.302975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:41967 (local address 127.8.64.190:42507)
0924 00:56:59.303196 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:59.303204 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:59.303487 (+ 283us) server_negotiation.cc:407] Connection header received
0924 00:56:59.303679 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:59.303684 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:59.303751 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:59.303849 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:59.304715 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:59.305430 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:59.306720 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:59.307135 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:59.307271 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:59.311661 (+ 4390us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.129:41967: BlockingRecv error: recv got EOF from 127.8.64.129:41967 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Sep 24 00:56:59 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.130@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:56:59 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.130@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:56:59.758859 17723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:56:59.750577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.130:58055)
0924 00:56:59.751059 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0924 00:56:59.751074 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 00:56:59.751226 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:56:59.751571 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:56:59.751575 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:56:59.752049 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:56:59.752230 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:59.752238 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:59.753269 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:59.753277 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:59.754242 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:56:59.754253 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:56:59.755050 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:56:59.755056 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:56:59.755230 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:59.755870 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:56:59.755889 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:56:59.758542 (+ 2653us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":262,"threads_started":1}
W20260924 00:56:59.758991 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:56:59.750606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:58055 (local address 127.8.64.190:42507)
0924 00:56:59.750837 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0924 00:56:59.750844 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:56:59.751161 (+ 317us) server_negotiation.cc:407] Connection header received
0924 00:56:59.751352 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:56:59.751357 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:56:59.751433 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:56:59.751543 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:56:59.752373 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:59.753130 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:59.754448 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:56:59.754868 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:56:59.755017 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:56:59.758735 (+ 3718us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.130:58055: BlockingRecv error: recv got EOF from 127.8.64.130:58055 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Sep 24 00:57:00 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.131@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:57:00 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.131@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:57:00.221066 17724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:00.208740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.131:60569)
0924 00:57:00.209371 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:00.209386 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:00.209539 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:00.209878 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:00.209882 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:00.210565 (+ 683us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:00.210756 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:00.210765 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:00.211763 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:00.211769 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:00.212818 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:00.212829 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:00.213556 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:00.213563 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:00.213736 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:00.217944 (+ 4208us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:57:00.217970 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:00.220847 (+ 2877us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":295,"threads_started":1}
W20260924 00:57:00.221324 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:57:00.208974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:60569 (local address 127.8.64.190:42507)
0924 00:57:00.209211 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:00.209216 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:00.209474 (+ 258us) server_negotiation.cc:407] Connection header received
0924 00:57:00.209662 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:00.209667 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:00.209740 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:00.209846 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:00.210897 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:00.211615 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:00.212968 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:00.213365 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:00.213519 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:00.221086 (+ 7567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.131:60569: BlockingRecv error: recv got EOF from 127.8.64.131:60569 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Sep 24 00:57:00 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.129@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:57:00 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.129@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:57:00.320752 17725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:00.312772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.129:56319)
0924 00:57:00.313315 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:00.313331 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:00.313473 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:00.313774 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:00.313777 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:00.314137 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:00.314274 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:00.314282 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:00.315249 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:00.315252 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:00.316145 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:00.316156 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:00.317102 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:00.317107 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:00.317216 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:00.317815 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:57:00.317832 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:00.320521 (+ 2689us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":310,"threads_started":1}
W20260924 00:57:00.320940 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:57:00.312823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:56319 (local address 127.8.64.190:42507)
0924 00:57:00.312994 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:00.312999 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:00.313415 (+ 416us) server_negotiation.cc:407] Connection header received
0924 00:57:00.313579 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:00.313582 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:00.313640 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:00.313721 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:00.314412 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:00.315137 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:00.316274 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:00.316926 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:00.317043 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:00.320699 (+ 3656us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.129:56319: BlockingRecv error: recv got EOF from 127.8.64.129:56319 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 24 00:57:00 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.130@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:57:00 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.130@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:57:00.767756 17726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:00.759821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.130:39397)
0924 00:57:00.760353 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:00.760368 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:00.760484 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:00.760750 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:00.760753 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:00.761142 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:00.761277 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:00.761285 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:00.762251 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:00.762255 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:00.763206 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:00.763217 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:00.763848 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:00.763852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:00.763959 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:00.764663 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:57:00.764679 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:00.767525 (+ 2846us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":315,"threads_started":1}
W20260924 00:57:00.767925 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:57:00.759968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:39397 (local address 127.8.64.190:42507)
0924 00:57:00.760147 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:00.760151 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:00.760442 (+ 291us) server_negotiation.cc:407] Connection header received
0924 00:57:00.760571 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:00.760574 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:00.760631 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:00.760711 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:00.761422 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:00.762134 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:00.763324 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:00.763725 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:00.763814 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:00.767722 (+ 3908us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.130:39397: BlockingRecv error: recv got EOF from 127.8.64.130:39397 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 24 00:57:01 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.131@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:57:01 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.131@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:57:01.230129 17727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:01.222067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.131:49937)
0924 00:57:01.222638 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:01.222655 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:01.222766 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:01.223106 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:01.223109 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:01.223500 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:01.223642 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:01.223650 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:01.224613 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:01.224617 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:01.225503 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:01.225515 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:01.226281 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:01.226285 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:01.226465 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:01.227048 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:57:01.227065 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:01.229897 (+ 2832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":331,"threads_started":1}
W20260924 00:57:01.230400 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:57:01.222236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:49937 (local address 127.8.64.190:42507)
0924 00:57:01.222452 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:01.222458 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:01.222734 (+ 276us) server_negotiation.cc:407] Connection header received
0924 00:57:01.222889 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:01.222893 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:01.222964 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:01.223048 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:01.223776 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:01.224494 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:01.225676 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:01.226111 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:01.226213 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:01.230087 (+ 3874us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.131:49937: BlockingRecv error: recv got EOF from 127.8.64.131:49937 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
Sep 24 00:57:01 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.129@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:57:01 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.129@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:57:01.329965 17728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:01.321705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.129:49993)
0924 00:57:01.322226 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:01.322242 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:01.322443 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:01.322779 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:01.322783 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:01.323147 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:01.323300 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:01.323308 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:01.324331 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:01.324336 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:01.325242 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:01.325253 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:01.325916 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:01.325921 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:01.326094 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:01.326738 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:57:01.326755 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:01.329725 (+ 2970us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":281,"threads_started":1}
W20260924 00:57:01.330120 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:57:01.321814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:49993 (local address 127.8.64.190:42507)
0924 00:57:01.322080 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:01.322086 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:01.322365 (+ 279us) server_negotiation.cc:407] Connection header received
0924 00:57:01.322565 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:01.322571 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:01.322646 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:01.322748 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:01.323444 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:01.324182 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:01.325382 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:01.325771 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:01.325902 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:01.329899 (+ 3997us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.129:49993: BlockingRecv error: recv got EOF from 127.8.64.129:49993 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Sep 24 00:57:01 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.130@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:57:01 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.130@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:57:01.777799 17729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:01.768753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.130:41629)
0924 00:57:01.769225 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:01.769242 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:01.769356 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:01.769751 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:01.769755 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:01.770107 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:01.770273 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:01.770281 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:01.771529 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:01.771533 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:01.772434 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:01.772444 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:01.773137 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:01.773142 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:01.773279 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:01.774282 (+ 1003us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:57:01.774363 (+ 81us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:01.777493 (+ 3130us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":274,"threads_started":1}
W20260924 00:57:01.777931 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:57:01.768819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:41629 (local address 127.8.64.190:42507)
0924 00:57:01.769049 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:01.769056 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:01.769322 (+ 266us) server_negotiation.cc:407] Connection header received
0924 00:57:01.769493 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:01.769504 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:01.769620 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:01.769702 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:01.770445 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:01.771404 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:01.772566 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:01.772952 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:01.773061 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:01.777696 (+ 4635us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.130:41629: BlockingRecv error: recv got EOF from 127.8.64.130:41629 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Sep 24 00:57:02 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.131@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:57:02 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.131@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:57:02.239398 17730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:02.231056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.131:33967)
0924 00:57:02.231630 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:02.231645 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:02.231806 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:02.232165 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:02.232170 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:02.232550 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:02.232708 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:02.232716 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:02.233810 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:02.233817 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:02.235008 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:02.235020 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:02.235699 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:02.235705 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:02.235882 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:02.236496 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:57:02.236512 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:02.239161 (+ 2649us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":362,"threads_started":1}
W20260924 00:57:02.239573 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:57:02.231109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:33967 (local address 127.8.64.190:42507)
0924 00:57:02.231412 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:02.231422 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:02.231738 (+ 316us) server_negotiation.cc:407] Connection header received
0924 00:57:02.231951 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:02.231955 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:02.232031 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:02.232136 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:02.232859 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:02.233687 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:02.235141 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:02.235558 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:02.235689 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:02.239349 (+ 3660us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.131:33967: BlockingRecv error: recv got EOF from 127.8.64.131:33967 (error 108)
Metrics: {"server-negotiator.queue_time_us":159}
Sep 24 00:57:02 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.129@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:57:02 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.129@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:57:02.338550 17731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:02.330838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.129:35951)
0924 00:57:02.331271 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:02.331287 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:02.331402 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:02.331696 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:02.331699 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:02.332052 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:02.332191 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:02.332199 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:02.333128 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:02.333132 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:02.334072 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:02.334082 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:02.334713 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:02.334717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:02.334901 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:02.335506 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:57:02.335522 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:02.338162 (+ 2640us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":253,"threads_started":1}
W20260924 00:57:02.338658 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:57:02.330917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:35951 (local address 127.8.64.190:42507)
0924 00:57:02.331179 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:02.331186 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:02.331384 (+ 198us) server_negotiation.cc:407] Connection header received
0924 00:57:02.331501 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:02.331505 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:02.331566 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:02.331647 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:02.332305 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:02.333023 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:02.334184 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:02.334598 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:02.334695 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:02.338387 (+ 3692us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.129:35951: BlockingRecv error: recv got EOF from 127.8.64.129:35951 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
Sep 24 00:57:02 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.130@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:57:02 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.130@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:57:02.786934 17732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:02.778801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.130:60115)
0924 00:57:02.779279 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:02.779295 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:02.779441 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:02.779783 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:02.779788 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:02.780169 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:02.780305 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:02.780313 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:02.781279 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:02.781286 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:02.782249 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:02.782259 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:02.782990 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:02.782996 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:02.783173 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:02.783707 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:57:02.783723 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:02.786600 (+ 2877us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":259,"threads_started":1}
W20260924 00:57:02.786993 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:57:02.778877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:60115 (local address 127.8.64.190:42507)
0924 00:57:02.779117 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:02.779125 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:02.779373 (+ 248us) server_negotiation.cc:407] Connection header received
0924 00:57:02.779580 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:02.779586 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:02.779655 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:02.779750 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:02.780437 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:02.781157 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:02.782414 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:02.782832 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:02.782971 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:02.786775 (+ 3804us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.130:60115: BlockingRecv error: recv got EOF from 127.8.64.130:60115 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Sep 24 00:57:03 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.131@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:57:03 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.131@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:57:03.248548 17733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:03.240374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.131:40631)
0924 00:57:03.240837 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:03.240854 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:03.240984 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:03.241304 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:03.241308 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:03.241675 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:03.241827 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:03.241836 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:03.242963 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:03.242966 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:03.243909 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:03.243921 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:03.244585 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:03.244589 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:03.244710 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:03.245317 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:57:03.245335 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:03.248315 (+ 2980us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":244,"threads_started":1}
W20260924 00:57:03.248737 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:57:03.240422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:40631 (local address 127.8.64.190:42507)
0924 00:57:03.240631 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:03.240641 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:03.240942 (+ 301us) server_negotiation.cc:407] Connection header received
0924 00:57:03.241083 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:03.241087 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:03.241150 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:03.241234 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:03.241984 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:03.242833 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:03.244035 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:03.244451 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:03.244550 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:03.248515 (+ 3965us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.131:40631: BlockingRecv error: recv got EOF from 127.8.64.131:40631 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 24 00:57:03 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.129@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:57:03 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.129@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:57:03.347779 17734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:03.339531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.129:34779)
0924 00:57:03.340035 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:03.340050 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:03.340197 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:03.340540 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:03.340544 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:03.340987 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:03.341141 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:03.341150 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:03.342147 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:03.342154 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:03.343126 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:03.343138 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:03.343827 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:03.343832 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:03.344013 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:03.344569 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:57:03.344586 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:03.347544 (+ 2958us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":254,"threads_started":1}
W20260924 00:57:03.347942 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:57:03.339583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:34779 (local address 127.8.64.190:42507)
0924 00:57:03.339814 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:03.339822 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:03.340131 (+ 309us) server_negotiation.cc:407] Connection header received
0924 00:57:03.340323 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:03.340327 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:03.340384 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:03.340506 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:03.341283 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:03.342018 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:03.343264 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:03.343679 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:03.343802 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:03.347718 (+ 3916us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.129:34779: BlockingRecv error: recv got EOF from 127.8.64.129:34779 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Sep 24 00:57:03 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.130@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:57:03 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.130@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:57:03.795681 17735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:03.787876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.130:34697)
0924 00:57:03.788406 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:03.788426 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:03.788531 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:03.788808 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:03.788811 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:03.789227 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:03.789437 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:03.789448 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:03.790401 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:03.790407 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:03.791287 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:03.791297 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:03.791928 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:03.791932 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:03.792046 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:03.792596 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:57:03.792613 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:03.795367 (+ 2754us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":261,"threads_started":1}
W20260924 00:57:03.795907 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:57:03.788016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:34697 (local address 127.8.64.190:42507)
0924 00:57:03.788176 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:03.788180 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:03.788494 (+ 314us) server_negotiation.cc:407] Connection header received
0924 00:57:03.788627 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:03.788630 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:03.788685 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:03.788763 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:03.789567 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:03.790280 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:03.791397 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:03.791811 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:03.791898 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:03.795647 (+ 3749us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.130:34697: BlockingRecv error: recv got EOF from 127.8.64.130:34697 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 24 00:57:04 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.131@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:57:04 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.131@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:57:04.257831 17736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:04.249532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.131:58431)
0924 00:57:04.250055 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:04.250072 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:04.250181 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:04.250550 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:04.250553 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:04.250922 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:04.251077 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:04.251085 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:04.252071 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:04.252075 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:04.253024 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:04.253035 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:04.253950 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:04.253955 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:04.254157 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:04.254870 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:57:04.254894 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:04.257587 (+ 2693us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":257,"threads_started":1}
W20260924 00:57:04.257957 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:57:04.249584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:58431 (local address 127.8.64.190:42507)
0924 00:57:04.249784 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:04.249789 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:04.250165 (+ 376us) server_negotiation.cc:407] Connection header received
0924 00:57:04.250337 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:04.250343 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:04.250419 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:04.250494 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:04.251189 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:04.251915 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:04.253164 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:04.253790 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:04.253943 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:04.257743 (+ 3800us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.131:58431: BlockingRecv error: recv got EOF from 127.8.64.131:58431 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Sep 24 00:57:04 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.129@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:57:04 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.129@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:57:04.356706 17737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:04.348645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.129:54421)
0924 00:57:04.349136 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:04.349151 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:04.349296 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:04.349612 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:04.349616 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:04.350035 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:04.350188 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:04.350197 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:04.351221 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:04.351226 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:04.352124 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:04.352135 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:04.352894 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:04.352900 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:04.353084 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:04.353682 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:57:04.353700 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:04.356368 (+ 2668us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":276,"threads_started":1}
W20260924 00:57:04.356801 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:57:04.348680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:54421 (local address 127.8.64.190:42507)
0924 00:57:04.348881 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:04.348888 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:04.349228 (+ 340us) server_negotiation.cc:407] Connection header received
0924 00:57:04.349413 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:04.349418 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:04.349488 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:04.349582 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:04.350345 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:04.351087 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:04.352272 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:04.352742 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:04.352862 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:04.356554 (+ 3692us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.129:54421: BlockingRecv error: recv got EOF from 127.8.64.129:54421 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 24 00:57:04 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.130@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:57:04 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.130@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:57:04.806878 17738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:04.796884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.130:60387)
0924 00:57:04.797451 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:04.797473 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:04.797618 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:04.797975 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:04.797980 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:04.798532 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:04.798720 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:04.798732 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:04.799796 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:04.799803 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:04.801430 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:04.801445 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:04.802459 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:04.802466 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:04.802666 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:04.803254 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:57:04.803285 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:04.806597 (+ 3312us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":290,"threads_started":1}
W20260924 00:57:04.807132 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:57:04.797049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:60387 (local address 127.8.64.190:42507)
0924 00:57:04.797197 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:04.797201 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:04.797591 (+ 390us) server_negotiation.cc:407] Connection header received
0924 00:57:04.797741 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:04.797745 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:04.797829 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:04.797918 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:04.798898 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:04.799659 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:04.801595 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:04.802242 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:04.802421 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:04.806875 (+ 4454us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.130:60387: BlockingRecv error: recv got EOF from 127.8.64.130:60387 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Sep 24 00:57:05 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.131@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:57:05 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.131@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:57:05.272187 17739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:05.258841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.131:60467)
0924 00:57:05.259361 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:05.259380 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:05.259558 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:05.259876 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:05.259880 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:05.260338 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:05.260543 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:05.260555 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:05.261551 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:05.261559 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:05.262762 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:05.262774 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:05.263544 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:05.263548 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:05.263667 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:05.268751 (+ 5084us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:57:05.268769 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:05.271861 (+ 3092us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":290,"threads_started":1}
W20260924 00:57:05.272324 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:57:05.258876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:60467 (local address 127.8.64.190:42507)
0924 00:57:05.259096 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:05.259105 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:05.259494 (+ 389us) server_negotiation.cc:407] Connection header received
0924 00:57:05.259681 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:05.259684 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:05.259745 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:05.259818 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:05.260673 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:05.261413 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:05.262918 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:05.263404 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:05.263529 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:05.272077 (+ 8548us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.131:60467: BlockingRecv error: recv got EOF from 127.8.64.131:60467 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Sep 24 00:57:05 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.129@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:57:05 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.129@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:57:05.365612 17740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:05.357627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.129:48277)
0924 00:57:05.358129 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:05.358146 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:05.358257 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:05.358590 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:05.358593 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:05.358974 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:05.359135 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:05.359143 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:05.360076 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:05.360080 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:05.360986 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:05.360996 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:05.361659 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:05.361664 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:05.361787 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:05.362393 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:57:05.362411 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:05.365191 (+ 2780us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":235,"threads_started":1}
W20260924 00:57:05.365701 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:57:05.357726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:48277 (local address 127.8.64.190:42507)
0924 00:57:05.357973 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:05.357979 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:05.358222 (+ 243us) server_negotiation.cc:407] Connection header received
0924 00:57:05.358390 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:05.358394 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:05.358452 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:05.358552 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:05.359250 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:05.359951 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:05.361100 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:05.361510 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:05.361600 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:05.365389 (+ 3789us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.129:48277: BlockingRecv error: recv got EOF from 127.8.64.129:48277 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Sep 24 00:57:05 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.130@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:57:05 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.130@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:57:05.816668 17741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:05.807806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.130:40071)
0924 00:57:05.808392 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:05.808435 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:05.808559 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:05.808975 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:05.808978 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:05.809366 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:05.809558 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:05.809567 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:05.810643 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:05.810646 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:05.811805 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:05.811817 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:05.812646 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:05.812653 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:05.812808 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:05.813455 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:57:05.813473 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:05.816304 (+ 2831us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":258,"threads_started":1}
W20260924 00:57:05.816731 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:57:05.807998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:40071 (local address 127.8.64.190:42507)
0924 00:57:05.808232 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:05.808238 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:05.808538 (+ 300us) server_negotiation.cc:407] Connection header received
0924 00:57:05.808736 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:05.808742 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:05.808812 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:05.808920 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:05.809704 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:05.810521 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:05.811932 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:05.812506 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:05.812606 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:05.816490 (+ 3884us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.130:40071: BlockingRecv error: recv got EOF from 127.8.64.130:40071 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Sep 24 00:57:06 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.131@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:57:06 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.131@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:57:06.281884 17742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:06.273115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.131:54927)
0924 00:57:06.273684 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:06.273701 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:06.273815 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:06.274169 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:06.274172 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:06.274592 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:06.274753 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:06.274762 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:06.275842 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:06.275848 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:06.277294 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:06.277315 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:06.278061 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:06.278065 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:06.278173 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:06.278800 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:57:06.278817 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:06.281633 (+ 2816us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":274,"threads_started":1}
W20260924 00:57:06.282039 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:57:06.273160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:54927 (local address 127.8.64.190:42507)
0924 00:57:06.273359 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:06.273368 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:06.273781 (+ 413us) server_negotiation.cc:407] Connection header received
0924 00:57:06.273962 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:06.273966 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:06.274031 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:06.274120 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:06.274914 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:06.275674 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:06.277447 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:06.277903 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:06.278011 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:06.281831 (+ 3820us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.131:54927: BlockingRecv error: recv got EOF from 127.8.64.131:54927 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Sep 24 00:57:06 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.129@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:57:06 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.129@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:57:06.375583 17743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:06.366779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.129:49411)
0924 00:57:06.367345 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:06.367362 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:06.367478 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:06.367772 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:06.367776 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:06.368163 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:06.368374 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:06.368382 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:06.369396 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:06.369401 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:06.370506 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:06.370518 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:06.371549 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:06.371554 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:06.371729 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:06.372438 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:57:06.372455 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:06.375350 (+ 2895us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":286,"threads_started":1}
W20260924 00:57:06.375792 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:57:06.366934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:49411 (local address 127.8.64.190:42507)
0924 00:57:06.367163 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:06.367167 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:06.367437 (+ 270us) server_negotiation.cc:407] Connection header received
0924 00:57:06.367574 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:06.367578 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:06.367638 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:06.367725 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:06.368500 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:06.369205 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:06.370671 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:06.371383 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:06.371537 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:06.375561 (+ 4024us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.129:49411: BlockingRecv error: recv got EOF from 127.8.64.129:49411 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Sep 24 00:57:06 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.130@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:57:06 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.130@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:57:06.825292 17744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:06.817635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.130:36607)
0924 00:57:06.818095 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:06.818111 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:06.818220 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:06.818538 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:06.818542 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:06.818915 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:06.819063 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:06.819071 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:06.820173 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:06.820177 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:06.821083 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:06.821093 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:06.821769 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:06.821773 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:06.821886 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:06.822551 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:57:06.822569 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:06.825073 (+ 2504us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":246,"threads_started":1}
W20260924 00:57:06.825459 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:57:06.817697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:36607 (local address 127.8.64.190:42507)
0924 00:57:06.817913 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:06.817919 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:06.818183 (+ 264us) server_negotiation.cc:407] Connection header received
0924 00:57:06.818354 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:06.818358 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:06.818418 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:06.818500 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:06.819179 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:06.820026 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:06.821210 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:06.821622 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:06.821713 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:06.825245 (+ 3532us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.130:36607: BlockingRecv error: recv got EOF from 127.8.64.130:36607 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 24 00:57:07 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.131@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:57:07 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.131@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:57:07.290588 17745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:07.283005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.131:35919)
0924 00:57:07.283570 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:07.283586 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:07.283697 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:07.284001 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:07.284004 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:07.284375 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:07.284524 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:07.284532 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:07.285445 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:07.285448 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:07.286360 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:07.286371 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:07.287043 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:07.287047 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:07.287160 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:07.287772 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:57:07.287788 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:07.290373 (+ 2585us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":318,"threads_started":1}
W20260924 00:57:07.290856 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:57:07.283085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:35919 (local address 127.8.64.190:42507)
0924 00:57:07.283278 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:07.283284 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:07.283645 (+ 361us) server_negotiation.cc:407] Connection header received
0924 00:57:07.283781 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:07.283785 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:07.283845 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:07.283924 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:07.284644 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:07.285321 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:07.286471 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:07.286897 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:07.286991 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:07.290596 (+ 3605us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.131:35919: BlockingRecv error: recv got EOF from 127.8.64.131:35919 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 24 00:57:07 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.129@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:57:07 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.129@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:57:07.385412 17746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:07.376568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.129:54795)
0924 00:57:07.377135 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:07.377151 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:07.377278 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:07.377625 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:07.377628 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:07.378146 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:07.378431 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:07.378442 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:07.379877 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:07.379881 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:07.380842 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:07.380853 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:07.381531 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:07.381535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:07.381649 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:07.382261 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:57:07.382279 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:07.385044 (+ 2765us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":320,"threads_started":1}
W20260924 00:57:07.385461 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:57:07.376611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:54795 (local address 127.8.64.190:42507)
0924 00:57:07.376809 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:07.376814 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:07.377240 (+ 426us) server_negotiation.cc:407] Connection header received
0924 00:57:07.377365 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:07.377368 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:07.377443 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:07.377550 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:07.378593 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:07.379732 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:07.380988 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:07.381393 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:07.381499 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:07.385236 (+ 3737us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.129:54795: BlockingRecv error: recv got EOF from 127.8.64.129:54795 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Sep 24 00:57:07 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.130@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:57:07 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.130@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:57:07.835016 17747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:07.826366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.130:43625)
0924 00:57:07.827003 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:07.827026 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:07.827151 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:07.827593 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:07.827598 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:07.828015 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:07.828286 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:07.828304 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:07.829371 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:07.829378 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:07.830400 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:07.830413 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:07.831161 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:07.831168 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:07.831353 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:07.831924 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:57:07.831942 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:07.834774 (+ 2832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":368,"threads_started":1}
W20260924 00:57:07.835259 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:57:07.826398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:43625 (local address 127.8.64.190:42507)
0924 00:57:07.826603 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:07.826607 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:07.827107 (+ 500us) server_negotiation.cc:407] Connection header received
0924 00:57:07.827320 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:07.827328 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:07.827419 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:07.827542 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:07.828443 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:07.829221 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:07.830553 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:07.830997 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:07.831138 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:07.835002 (+ 3864us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.130:43625: BlockingRecv error: recv got EOF from 127.8.64.130:43625 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Sep 24 00:57:08 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.131@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:57:08 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.131@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:57:08.299698 17748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:08.291612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.131:46951)
0924 00:57:08.292335 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:08.292351 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:08.292480 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:08.292762 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:08.292765 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:08.293124 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:08.293278 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:08.293285 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:08.294192 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:08.294195 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:08.295187 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:08.295201 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:08.295912 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:08.295916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:08.296055 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:08.296577 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:57:08.296593 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:08.299463 (+ 2870us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":635,"thread_start_us":490,"threads_started":1}
W20260924 00:57:08.299850 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:57:08.291663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:46951 (local address 127.8.64.190:42507)
0924 00:57:08.291972 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:08.291978 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:08.292436 (+ 458us) server_negotiation.cc:407] Connection header received
0924 00:57:08.292579 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:08.292583 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:08.292643 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:08.292723 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:08.293390 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:08.294089 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:08.295322 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:08.295788 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:08.295888 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:08.299627 (+ 3739us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.131:46951: BlockingRecv error: recv got EOF from 127.8.64.131:46951 (error 108)
Metrics: {"server-negotiator.queue_time_us":191}
Sep 24 00:57:08 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.129@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:57:08 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.129@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:57:08.394639 17749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:08.386398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.129:40547)
0924 00:57:08.386913 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:08.386942 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:08.387051 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:08.387333 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:08.387336 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:08.387692 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:08.387827 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:08.387834 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:08.388794 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:08.388798 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:08.389701 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:08.389712 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:08.390447 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:08.390450 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:08.390565 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:08.391234 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:57:08.391260 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:08.394269 (+ 3009us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":291,"threads_started":1}
W20260924 00:57:08.394735 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:57:08.386472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:40547 (local address 127.8.64.190:42507)
0924 00:57:08.386632 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:08.386637 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:08.387015 (+ 378us) server_negotiation.cc:407] Connection header received
0924 00:57:08.387140 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:08.387144 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:08.387214 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:08.387299 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:08.387950 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:08.388681 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:08.389821 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:08.390240 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:08.390425 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:08.394492 (+ 4067us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.129:40547: BlockingRecv error: recv got EOF from 127.8.64.129:40547 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 24 00:57:08 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.130@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:57:08 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.130@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:57:08.844067 17750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:08.835992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.130:53575)
0924 00:57:08.836464 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:08.836480 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:08.836584 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:08.836846 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:08.836849 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:08.837193 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:08.837333 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:08.837341 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:08.838484 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:08.838489 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:08.839655 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:08.839667 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:08.840323 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:08.840327 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:08.840444 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:08.840992 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:57:08.841008 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:08.843853 (+ 2845us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":264,"threads_started":1}
W20260924 00:57:08.844319 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:57:08.836071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:53575 (local address 127.8.64.190:42507)
0924 00:57:08.836307 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:08.836313 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:08.836557 (+ 244us) server_negotiation.cc:407] Connection header received
0924 00:57:08.836671 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:08.836674 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:08.836733 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:08.836809 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:08.837448 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:08.838329 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:08.839777 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:08.840194 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:08.840295 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:08.844056 (+ 3761us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.130:53575: BlockingRecv error: recv got EOF from 127.8.64.130:53575 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Sep 24 00:57:09 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.131@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:57:09 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.131@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:57:09.308745 17751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:09.300559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.131:41547)
0924 00:57:09.301273 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:09.301292 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:09.301425 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:09.301894 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:09.301898 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:09.302444 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:09.302623 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:09.302633 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:09.303603 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:09.303606 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:09.304509 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:09.304519 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:09.305123 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:09.305127 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:09.305243 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:09.305813 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:57:09.305829 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:09.308531 (+ 2702us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":486,"threads_started":1}
W20260924 00:57:09.308909 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:57:09.300816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:41547 (local address 127.8.64.190:42507)
0924 00:57:09.301030 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:09.301038 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:09.301356 (+ 318us) server_negotiation.cc:407] Connection header received
0924 00:57:09.301581 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:09.301586 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:09.301645 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:09.301866 (+ 221us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:09.302748 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:09.303480 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:09.304622 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:09.305005 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:09.305094 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:09.308707 (+ 3613us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.131:41547: BlockingRecv error: recv got EOF from 127.8.64.131:41547 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Sep 24 00:57:09 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.129@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:57:09 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.129@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:57:09.403865 17752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:09.395682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.129:34807)
0924 00:57:09.396242 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:09.396263 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:09.396401 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:09.396787 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:09.396791 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:09.397319 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:09.397512 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:09.397522 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:09.398549 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:09.398553 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:09.399421 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:09.399431 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:09.400095 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:09.400102 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:09.400289 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:09.400935 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:57:09.400953 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:09.403634 (+ 2681us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":278,"threads_started":1}
W20260924 00:57:09.404021 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:57:09.395748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:34807 (local address 127.8.64.190:42507)
0924 00:57:09.395969 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:09.395977 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:09.396348 (+ 371us) server_negotiation.cc:407] Connection header received
0924 00:57:09.396540 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:09.396545 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:09.396645 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:09.396736 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:09.397639 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:09.398422 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:09.399535 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:09.399961 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:09.400052 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:09.403803 (+ 3751us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.129:34807: BlockingRecv error: recv got EOF from 127.8.64.129:34807 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 24 00:57:09 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.130@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:57:09 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.130@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:57:09.853406 17753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:09.844977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.130:40073)
0924 00:57:09.845706 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:09.845725 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:09.845844 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:09.846190 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:09.846194 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:09.846726 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:09.846903 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:09.846912 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:09.847953 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:09.847959 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:09.848837 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:09.848848 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:09.849586 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:09.849592 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:09.849762 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:09.850333 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:57:09.850350 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:09.853169 (+ 2819us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":532,"threads_started":1}
W20260924 00:57:09.853569 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:57:09.845071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:40073 (local address 127.8.64.190:42507)
0924 00:57:09.845256 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:09.845263 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:09.845818 (+ 555us) server_negotiation.cc:407] Connection header received
0924 00:57:09.845967 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:09.845974 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:09.846052 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:09.846154 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:09.847054 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:09.847813 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:09.848988 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:09.849428 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:09.849557 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:09.853344 (+ 3787us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.130:40073: BlockingRecv error: recv got EOF from 127.8.64.130:40073 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 24 00:57:10 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.131@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:57:10 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.131@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:57:10.323609 17754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:10.309596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.131:36255)
0924 00:57:10.310159 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:10.310186 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:10.310357 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:10.310685 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:10.310688 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:10.311078 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:10.311251 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:10.311260 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:10.312284 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:10.312288 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:10.313191 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:10.313203 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:10.313890 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:10.313894 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:10.314003 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:10.320082 (+ 6079us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:57:10.320110 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:10.323346 (+ 3236us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":326,"threads_started":1}
W20260924 00:57:10.323757 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:57:10.309669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:36255 (local address 127.8.64.190:42507)
0924 00:57:10.309820 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:10.309824 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:10.310253 (+ 429us) server_negotiation.cc:407] Connection header received
0924 00:57:10.310470 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:10.310474 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:10.310549 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:10.310628 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:10.311413 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:10.312169 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:10.313335 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:10.313730 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:10.313847 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:10.323538 (+ 9691us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.131:36255: BlockingRecv error: recv got EOF from 127.8.64.131:36255 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 24 00:57:10 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.129@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:57:10 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.129@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:57:10.413491 17755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:10.404798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.129:54455)
0924 00:57:10.405601 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:10.405616 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:10.405731 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:10.406047 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:10.406050 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:10.406547 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:10.406693 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:10.406702 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:10.407735 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:10.407739 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:10.408733 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:10.408745 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:10.409531 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:10.409535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:10.409655 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:10.410357 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:57:10.410380 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:10.413133 (+ 2753us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":694,"thread_start_us":572,"threads_started":1}
W20260924 00:57:10.413559 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:57:10.404877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:54455 (local address 127.8.64.190:42507)
0924 00:57:10.405160 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:10.405169 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:10.405692 (+ 523us) server_negotiation.cc:407] Connection header received
0924 00:57:10.405824 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:10.405828 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:10.405901 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:10.405985 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:10.406819 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:10.407615 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:10.408852 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:10.409406 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:10.409508 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:10.413322 (+ 3814us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.129:54455: BlockingRecv error: recv got EOF from 127.8.64.129:54455 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Sep 24 00:57:10 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.130@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:57:10 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.130@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:57:10.862540 17756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:10.854370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.130:38157)
0924 00:57:10.854910 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:10.854926 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:10.855040 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:10.855380 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:10.855384 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:10.855755 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:10.855907 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:10.855915 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:10.856960 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:10.856964 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:10.857937 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:10.857948 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:10.858659 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:10.858663 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:10.858799 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:10.859378 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:57:10.859396 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:10.862185 (+ 2789us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":315,"threads_started":1}
W20260924 00:57:10.862687 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:57:10.854574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:38157 (local address 127.8.64.190:42507)
0924 00:57:10.854751 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:10.854756 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:10.854996 (+ 240us) server_negotiation.cc:407] Connection header received
0924 00:57:10.855180 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:10.855183 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:10.855241 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:10.855326 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:10.856063 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:10.856794 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:10.858074 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:10.858544 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:10.858636 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:10.862443 (+ 3807us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.130:38157: BlockingRecv error: recv got EOF from 127.8.64.130:38157 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 24 00:57:11 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.131@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:57:11 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.131@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:57:11.332671 17757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:11.324519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.131:44585)
0924 00:57:11.325053 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:11.325069 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:11.325178 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:11.325429 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:11.325431 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:11.325816 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:11.325967 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:11.325976 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:11.326973 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:11.326978 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:11.327873 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:11.327884 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:11.328520 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:11.328523 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:11.328642 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:11.329229 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:57:11.329252 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:11.332315 (+ 3063us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":294,"threads_started":1}
W20260924 00:57:11.332774 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:57:11.324578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:44585 (local address 127.8.64.190:42507)
0924 00:57:11.324746 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:11.324750 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:11.325142 (+ 392us) server_negotiation.cc:407] Connection header received
0924 00:57:11.325273 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:11.325277 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:11.325336 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:11.325416 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:11.326090 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:11.326850 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:11.327984 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:11.328395 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:11.328484 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:11.332535 (+ 4051us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.131:44585: BlockingRecv error: recv got EOF from 127.8.64.131:44585 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Sep 24 00:57:11 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.129@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:57:11 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.129@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:57:11.422629 17758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:11.414511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.129:52653)
0924 00:57:11.415025 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:11.415040 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:11.415192 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:11.415537 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:11.415541 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:11.415935 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:11.416102 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:11.416110 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:11.417110 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:11.417116 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:11.418046 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:11.418057 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:11.418873 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:11.418880 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:11.419043 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:11.419592 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:57:11.419608 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:11.422255 (+ 2647us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":278,"threads_started":1}
W20260924 00:57:11.422731 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:57:11.414562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:52653 (local address 127.8.64.190:42507)
0924 00:57:11.414756 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:11.414765 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:11.415120 (+ 355us) server_negotiation.cc:407] Connection header received
0924 00:57:11.415318 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:11.415322 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:11.415392 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:11.415520 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:11.416237 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:11.416977 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:11.418186 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:11.418692 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:11.418838 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:11.422490 (+ 3652us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.129:52653: BlockingRecv error: recv got EOF from 127.8.64.129:52653 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 24 00:57:11 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.130@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:57:11 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.130@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:57:11.871672 17759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:11.863555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.130:43105)
0924 00:57:11.864093 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:11.864108 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:11.864254 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:11.864598 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:11.864603 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:11.864995 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:11.865150 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:11.865158 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:11.866156 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:11.866163 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:11.867105 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:11.867116 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:11.867838 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:11.867845 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:11.868022 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:11.868622 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:57:11.868639 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:11.871431 (+ 2792us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":264,"threads_started":1}
W20260924 00:57:11.871816 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:57:11.863633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:43105 (local address 127.8.64.190:42507)
0924 00:57:11.863798 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:11.863803 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:11.864194 (+ 391us) server_negotiation.cc:407] Connection header received
0924 00:57:11.864387 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:11.864392 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:11.864457 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:11.864575 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:11.865293 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:11.866027 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:11.867246 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:11.867661 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:11.867797 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:11.871620 (+ 3823us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.130:43105: BlockingRecv error: recv got EOF from 127.8.64.130:43105 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 24 00:57:12 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.131@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:57:12 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.131@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:57:12.342555 17760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:12.333618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.131:32803)
0924 00:57:12.334196 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:12.334212 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:12.334343 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:12.334633 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:12.334636 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:12.335004 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:12.335152 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:12.335161 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:12.336528 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:12.336539 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:12.338012 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:12.338025 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:12.338766 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:12.338772 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:12.338938 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:12.339487 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:57:12.339503 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:12.342262 (+ 2759us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":292,"threads_started":1}
W20260924 00:57:12.342681 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:57:12.333679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:32803 (local address 127.8.64.190:42507)
0924 00:57:12.333830 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:12.333834 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:12.334280 (+ 446us) server_negotiation.cc:407] Connection header received
0924 00:57:12.334439 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:12.334443 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:12.334507 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:12.334592 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:12.335267 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:12.336373 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:12.338149 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:12.338588 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:12.338723 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:12.342477 (+ 3754us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.131:32803: BlockingRecv error: recv got EOF from 127.8.64.131:32803 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 24 00:57:12 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.129@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:57:12 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.129@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:57:12.431892 17761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:12.423656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.129:56143)
0924 00:57:12.424143 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:12.424166 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:12.424315 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:12.424647 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:12.424651 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:12.425049 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:12.425214 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:12.425223 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:12.426265 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:12.426270 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:12.427203 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:12.427213 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:12.427901 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:12.427907 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:12.428086 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:12.428675 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:57:12.428692 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:12.431563 (+ 2871us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":258,"threads_started":1}
W20260924 00:57:12.432111 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:57:12.423715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:56143 (local address 127.8.64.190:42507)
0924 00:57:12.424062 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:12.424069 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:12.424245 (+ 176us) server_negotiation.cc:407] Connection header received
0924 00:57:12.424444 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:12.424449 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:12.424521 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:12.424617 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:12.425354 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:12.426121 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:12.427337 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:12.427753 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:12.427878 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:12.431831 (+ 3953us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.129:56143: BlockingRecv error: recv got EOF from 127.8.64.129:56143 (error 108)
Metrics: {"server-negotiator.queue_time_us":197}
Sep 24 00:57:12 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.130@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:57:12 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.130@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:57:12.881330 17762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:12.872711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.130:40309)
0924 00:57:12.873217 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:12.873233 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:12.873390 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:12.873732 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:12.873736 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:12.874122 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:12.874329 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:12.874343 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:12.875343 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:12.875348 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:12.876578 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:12.876589 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:12.877328 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:12.877332 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:12.877453 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:12.878139 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:57:12.878156 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:12.881079 (+ 2923us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":288,"threads_started":1}
W20260924 00:57:12.881541 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:57:12.872826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:40309 (local address 127.8.64.190:42507)
0924 00:57:12.873082 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:12.873089 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:12.873314 (+ 225us) server_negotiation.cc:407] Connection header received
0924 00:57:12.873528 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:12.873540 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:12.873602 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:12.873705 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:12.874470 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:12.875207 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:12.876744 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:12.877197 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:12.877294 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:12.881319 (+ 4025us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.130:40309: BlockingRecv error: recv got EOF from 127.8.64.130:40309 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Sep 24 00:57:13 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.131@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:57:13 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.131@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:57:13.351073 17763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:13.343496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.131:50455)
0924 00:57:13.343930 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:13.343959 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:13.344068 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:13.344342 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:13.344345 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:13.344696 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:13.344858 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:13.344867 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:13.345807 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:13.345810 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:13.346807 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:13.346818 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:13.347506 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:13.347510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:13.347617 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:13.348205 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:57:13.348222 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:13.350826 (+ 2604us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":237,"threads_started":1}
W20260924 00:57:13.351236 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:57:13.343581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:50455 (local address 127.8.64.190:42507)
0924 00:57:13.343833 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:13.343840 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:13.344040 (+ 200us) server_negotiation.cc:407] Connection header received
0924 00:57:13.344162 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:13.344166 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:13.344227 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:13.344305 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:13.344975 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:13.345688 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:13.346921 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:13.347390 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:13.347481 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:13.351038 (+ 3557us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.131:50455: BlockingRecv error: recv got EOF from 127.8.64.131:50455 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Sep 24 00:57:13 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.129@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:57:13 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.129@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:57:13.441685 17764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:13.433169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.129:41461)
0924 00:57:13.433700 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:13.433716 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:13.433826 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:13.434147 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:13.434150 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:13.434613 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:13.434803 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:13.434812 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:13.435948 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:13.435956 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:13.437089 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:13.437102 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:13.437856 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:13.437860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:13.437977 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:13.438690 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:57:13.438708 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:13.441429 (+ 2721us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":302,"threads_started":1}
W20260924 00:57:13.441903 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:57:13.433322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:41461 (local address 127.8.64.190:42507)
0924 00:57:13.433508 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:13.433513 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:13.433789 (+ 276us) server_negotiation.cc:407] Connection header received
0924 00:57:13.433923 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:13.433927 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:13.434020 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:13.434110 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:13.434938 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:13.435789 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:13.437217 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:13.437729 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:13.437827 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:13.441643 (+ 3816us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.129:41461: BlockingRecv error: recv got EOF from 127.8.64.129:41461 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Sep 24 00:57:13 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.130@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:57:13 dist-test-slave-mmhs krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.64.130@KRBTEST.COM for oryx/127.8.64.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:57:13.890642 17765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:13.882486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42507 (local address 127.8.64.130:47111)
0924 00:57:13.882930 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:13.882946 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:13.883055 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:13.883356 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:13.883359 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:13.883759 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:13.883938 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:13.883946 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:13.884893 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:13.884897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:13.885788 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:13.885798 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:13.886617 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:13.886624 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:13.886798 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:13.887502 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:57:13.887548 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:13.890394 (+ 2846us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.64.190:42507: Server oryx/127.8.64.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":247,"threads_started":1}
W20260924 00:57:13.890813 17455 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:57:13.882527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:47111 (local address 127.8.64.190:42507)
0924 00:57:13.882747 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:13.882757 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:13.883041 (+ 284us) server_negotiation.cc:407] Connection header received
0924 00:57:13.883159 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:13.883163 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:13.883231 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:13.883334 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:13.884081 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:13.884773 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:13.885903 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:13.886437 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:13.886578 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:13.890573 (+ 3995us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.64.130:47111: BlockingRecv error: recv got EOF from 127.8.64.130:47111 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260924 00:57:14.180642 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 17304
I20260924 00:57:14.192305 17767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:14.181638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47750 (local address 127.8.64.129:46095)
0924 00:57:14.182157 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:14.182165 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:14.182200 (+ 35us) server_negotiation.cc:407] Connection header received
0924 00:57:14.182263 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:14.182270 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:14.182569 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:14.182739 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:14.184090 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:14.185083 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:14.186225 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:14.186813 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:14.186911 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:14.189246 (+ 2335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:57:14.189254 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:57:14.189267 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:57:14.189302 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:57:14.191853 (+ 2551us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.64.129@KRBTEST.COM not found in keytab (ticket kvno 2)
0924 00:57:14.192092 (+ 239us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47750: Request ticket server kudu/127.8.64.129@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":259,"threads_started":1}
W20260924 00:57:14.192428 17767 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47750: Request ticket server kudu/127.8.64.129@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260924 00:57:14.192566 8450 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 17304: Not authorized: Client connection negotiation failed: client connection to 127.8.64.129:46095: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.64.129@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260924 00:57:14.192782 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 17304
I20260924 00:57:14.209758 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 17457
I20260924 00:57:14.221452 17768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:14.210772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52898 (local address 127.8.64.130:41613)
0924 00:57:14.211305 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:14.211313 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:14.211342 (+ 29us) server_negotiation.cc:407] Connection header received
0924 00:57:14.211399 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:14.211405 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:14.211616 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:14.211803 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:14.213110 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:14.214123 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:14.215402 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:14.215946 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:14.216107 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:14.218550 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:57:14.218559 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:57:14.218572 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:57:14.218605 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:57:14.220801 (+ 2196us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.64.130@KRBTEST.COM not found in keytab (ticket kvno 2)
0924 00:57:14.221129 (+ 328us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52898: Request ticket server kudu/127.8.64.130@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":259,"threads_started":1}
W20260924 00:57:14.221487 8450 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 17457: Not authorized: Client connection negotiation failed: client connection to 127.8.64.130:41613: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.64.130@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260924 00:57:14.221562 17768 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52898: Request ticket server kudu/127.8.64.130@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260924 00:57:14.221647 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 17457
I20260924 00:57:14.238029 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 17588
I20260924 00:57:14.250078 17769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:14.238940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51842 (local address 127.8.64.131:43379)
0924 00:57:14.239452 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:14.239462 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:14.239491 (+ 29us) server_negotiation.cc:407] Connection header received
0924 00:57:14.239547 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:14.239553 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:14.239772 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:14.239970 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:14.241246 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:14.242478 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:14.243673 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:14.244201 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:14.244351 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:14.246902 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:57:14.246911 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:57:14.246924 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:57:14.246960 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:57:14.249388 (+ 2428us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.64.131@KRBTEST.COM not found in keytab (ticket kvno 2)
0924 00:57:14.249715 (+ 327us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51842: Request ticket server kudu/127.8.64.131@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":262,"threads_started":1}
W20260924 00:57:14.250054 8450 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 17588: Not authorized: Client connection negotiation failed: client connection to 127.8.64.131:43379: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.64.131@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260924 00:57:14.250168 17769 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51842: Request ticket server kudu/127.8.64.131@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260924 00:57:14.250219 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 17588
I20260924 00:57:14.266187 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 17235
I20260924 00:57:14.267541 17297 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:57:14.378852 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 17235
2026-09-24T00:57:14Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21045 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-24T00:57:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:57:14Z Disabled control of system clock
I20260924 00:57:14.455212 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:42547
--webserver_interface=127.8.64.190
--webserver_port=0
--builtin_ntp_servers=127.8.64.148:43121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:42547
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260924 00:57:14.776839 17774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:57:14.777117 17774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:57:14.777194 17774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:57:14.784256 17774 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:57:14.784363 17774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:57:14.784407 17774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:57:14.784466 17774 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:57:14.784508 17774 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:57:14.793293 17774 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:43121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:42547
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:42547
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:57:14.794713 17774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:57:14.796543 17774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:57:14.808688 17779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:57:14.808765 17780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:57:14.808660 17782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:57:14.809408 17774 server_base.cc:1034] running on GCE node
I20260924 00:57:14.810226 17774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:57:14.811621 17774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:57:14.812829 17774 hybrid_clock.cc:648] HybridClock initialized: now 1790211434812788 us; error 32 us; skew 500 ppm
I20260924 00:57:14.813194 17774 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 00:57:14.815512 17774 webserver.cc:460] Webserver started at http://127.8.64.190:44765/ using document root <none> and password file <none>
I20260924 00:57:14.816227 17774 fs_manager.cc:359] Metadata directory not provided
I20260924 00:57:14.816341 17774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:57:14.816684 17774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:57:14.819577 17774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "9f1844ee308a424893a778cc65bf51e4"
format_stamp: "Formatted at 2026-09-24 00:57:14 on dist-test-slave-mmhs"
server_key: "70fd0c0595f6a07e999b9643a4e7a4df"
server_key_iv: "78a41003777141e0f3ea6f77f7701ace"
server_key_version: "encryptionkey@0"
I20260924 00:57:14.820371 17774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "9f1844ee308a424893a778cc65bf51e4"
format_stamp: "Formatted at 2026-09-24 00:57:14 on dist-test-slave-mmhs"
server_key: "70fd0c0595f6a07e999b9643a4e7a4df"
server_key_iv: "78a41003777141e0f3ea6f77f7701ace"
server_key_version: "encryptionkey@0"
I20260924 00:57:14.825472 17774 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260924 00:57:14.828866 17788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:57:14.830194 17774 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260924 00:57:14.830395 17774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "9f1844ee308a424893a778cc65bf51e4"
format_stamp: "Formatted at 2026-09-24 00:57:14 on dist-test-slave-mmhs"
server_key: "70fd0c0595f6a07e999b9643a4e7a4df"
server_key_iv: "78a41003777141e0f3ea6f77f7701ace"
server_key_version: "encryptionkey@0"
I20260924 00:57:14.830616 17774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:57:14.844779 17774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:57:14.845561 17774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:57:14.845858 17774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:57:14.863281 17774 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.190:42547
I20260924 00:57:14.863298 17839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.190:42547 every 8 connection(s)
I20260924 00:57:14.864610 17774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 00:57:14.865056 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 17774
I20260924 00:57:14.865273 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 00:57:14.866074 8450 external_mini_cluster.cc:1428] Setting key 5ad7262fbfdc8a54b3b1bc698ecd8ef5
I20260924 00:57:14.869081 17840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:57:14.879014 17843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:14.868101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35842 (local address 127.8.64.190:42547)
0924 00:57:14.868993 (+ 892us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:14.869003 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:14.869049 (+ 46us) server_negotiation.cc:407] Connection header received
0924 00:57:14.870696 (+ 1647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:14.870717 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:14.871145 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:14.871529 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:14.872094 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:14.873369 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:14.874724 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:14.875362 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:14.875557 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:14.876777 (+ 1220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:57:14.876785 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:57:14.876797 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:57:14.877353 (+ 556us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:57:14.877421 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:57:14.877443 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:57:14.877593 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0924 00:57:14.877800 (+ 207us) server_negotiation.cc:299] Negotiation successful
0924 00:57:14.878077 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":513,"thread_start_us":311,"threads_started":1}
I20260924 00:57:14.879668 17840 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f1844ee308a424893a778cc65bf51e4: Bootstrap starting.
I20260924 00:57:14.883196 17840 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f1844ee308a424893a778cc65bf51e4: Neither blocks nor log segments found. Creating new log.
I20260924 00:57:14.884146 17840 log.cc:824] T 00000000000000000000000000000000 P 9f1844ee308a424893a778cc65bf51e4: Log is configured to *not* fsync() on all Append() calls
I20260924 00:57:14.886961 17840 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f1844ee308a424893a778cc65bf51e4: No bootstrap required, opened a new log
I20260924 00:57:14.893996 17840 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f1844ee308a424893a778cc65bf51e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f1844ee308a424893a778cc65bf51e4" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42547 } }
I20260924 00:57:14.894390 17840 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f1844ee308a424893a778cc65bf51e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:57:14.894498 17840 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f1844ee308a424893a778cc65bf51e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1844ee308a424893a778cc65bf51e4, State: Initialized, Role: FOLLOWER
I20260924 00:57:14.895052 17840 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f1844ee308a424893a778cc65bf51e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f1844ee308a424893a778cc65bf51e4" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42547 } }
I20260924 00:57:14.895272 17840 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f1844ee308a424893a778cc65bf51e4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:57:14.895404 17840 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f1844ee308a424893a778cc65bf51e4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:57:14.895603 17840 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f1844ee308a424893a778cc65bf51e4 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:57:14.897697 17840 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f1844ee308a424893a778cc65bf51e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f1844ee308a424893a778cc65bf51e4" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42547 } }
I20260924 00:57:14.898178 17840 leader_election.cc:302] T 00000000000000000000000000000000 P 9f1844ee308a424893a778cc65bf51e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f1844ee308a424893a778cc65bf51e4; no voters:
I20260924 00:57:14.898845 17840 leader_election.cc:288] T 00000000000000000000000000000000 P 9f1844ee308a424893a778cc65bf51e4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:57:14.898994 17845 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f1844ee308a424893a778cc65bf51e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:57:14.899735 17845 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f1844ee308a424893a778cc65bf51e4 [term 1 LEADER]: Becoming Leader. State: Replica: 9f1844ee308a424893a778cc65bf51e4, State: Running, Role: LEADER
I20260924 00:57:14.900259 17845 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f1844ee308a424893a778cc65bf51e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f1844ee308a424893a778cc65bf51e4" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42547 } }
I20260924 00:57:14.901015 17840 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f1844ee308a424893a778cc65bf51e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:57:14.903200 17847 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f1844ee308a424893a778cc65bf51e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f1844ee308a424893a778cc65bf51e4. Latest consensus state: current_term: 1 leader_uuid: "9f1844ee308a424893a778cc65bf51e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f1844ee308a424893a778cc65bf51e4" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42547 } } }
I20260924 00:57:14.903628 17847 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f1844ee308a424893a778cc65bf51e4 [sys.catalog]: This master's current role is: LEADER
I20260924 00:57:14.904152 17846 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f1844ee308a424893a778cc65bf51e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f1844ee308a424893a778cc65bf51e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f1844ee308a424893a778cc65bf51e4" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42547 } } }
I20260924 00:57:14.904451 17846 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f1844ee308a424893a778cc65bf51e4 [sys.catalog]: This master's current role is: LEADER
I20260924 00:57:14.905443 17854 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:57:14.910339 17854 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:57:14.919500 17854 catalog_manager.cc:1347] Generated new cluster ID: 794809e53e3d427bb187d69d7707df1b
I20260924 00:57:14.919629 17854 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:57:14.941435 17854 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:57:14.942749 17854 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:57:14.950824 17854 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f1844ee308a424893a778cc65bf51e4: Generated new TSK 0
I20260924 00:57:14.951680 17854 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 00:57:14.964288 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.129:0
--local_ip_for_outbound_sockets=127.8.64.129
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:42547
--builtin_ntp_servers=127.8.64.148:43121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260924 00:57:15.321751 17864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:57:15.322084 17864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:57:15.322235 17864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:57:15.329802 17864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:57:15.329943 17864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:57:15.330122 17864 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.129
I20260924 00:57:15.339461 17864 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:43121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:42547
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.129
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:57:15.340745 17864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:57:15.342659 17864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:57:15.356480 17869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:57:15.356484 17870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:57:15.356530 17872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:57:15.357172 17864 server_base.cc:1034] running on GCE node
I20260924 00:57:15.357816 17864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:57:15.358901 17864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:57:15.360158 17864 hybrid_clock.cc:648] HybridClock initialized: now 1790211435360094 us; error 54 us; skew 500 ppm
I20260924 00:57:15.360518 17864 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 00:57:15.370908 17864 webserver.cc:460] Webserver started at http://127.8.64.129:34547/ using document root <none> and password file <none>
I20260924 00:57:15.371714 17864 fs_manager.cc:359] Metadata directory not provided
I20260924 00:57:15.371869 17864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:57:15.372233 17864 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:57:15.375272 17864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/data/instance:
uuid: "bd5dc9a6292a4f94be437b27ab9bc368"
format_stamp: "Formatted at 2026-09-24 00:57:15 on dist-test-slave-mmhs"
server_key: "cdb6072d8e77619bd3810f0f8b582fae"
server_key_iv: "bc7e5441b98bb8789ff9393d669e0326"
server_key_version: "encryptionkey@0"
I20260924 00:57:15.376129 17864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance:
uuid: "bd5dc9a6292a4f94be437b27ab9bc368"
format_stamp: "Formatted at 2026-09-24 00:57:15 on dist-test-slave-mmhs"
server_key: "cdb6072d8e77619bd3810f0f8b582fae"
server_key_iv: "bc7e5441b98bb8789ff9393d669e0326"
server_key_version: "encryptionkey@0"
I20260924 00:57:15.381562 17864 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260924 00:57:15.385015 17879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:57:15.386417 17864 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 00:57:15.386580 17864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/wal
uuid: "bd5dc9a6292a4f94be437b27ab9bc368"
format_stamp: "Formatted at 2026-09-24 00:57:15 on dist-test-slave-mmhs"
server_key: "cdb6072d8e77619bd3810f0f8b582fae"
server_key_iv: "bc7e5441b98bb8789ff9393d669e0326"
server_key_version: "encryptionkey@0"
I20260924 00:57:15.386785 17864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:57:15.410594 17864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:57:15.411465 17864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:57:15.411770 17864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:57:15.412770 17864 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:57:15.414249 17864 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:57:15.414384 17864 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260924 00:57:15.414515 17864 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:57:15.414600 17864 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:57:15.447804 17864 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.129:44623
I20260924 00:57:15.447893 17992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.129:44623 every 8 connection(s)
I20260924 00:57:15.449143 17864 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
I20260924 00:57:15.453797 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 17864
I20260924 00:57:15.453991 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance
I20260924 00:57:15.454808 8450 external_mini_cluster.cc:1428] Setting key e79c2d07a45d4bb1f9ab2525a1720584
I20260924 00:57:15.458012 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.130:0
--local_ip_for_outbound_sockets=127.8.64.130
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:42547
--builtin_ntp_servers=127.8.64.148:43121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260924 00:57:15.464058 17995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:15.451866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:51743 (local address 127.8.64.190:42547)
0924 00:57:15.452308 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:15.452312 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:15.453474 (+ 1162us) server_negotiation.cc:407] Connection header received
0924 00:57:15.456108 (+ 2634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:15.456118 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:15.456221 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:15.456337 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:15.457669 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:15.458485 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:15.461000 (+ 2515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:15.461414 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:15.461538 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:15.463280 (+ 1742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:57:15.463285 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:57:15.463289 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:57:15.463307 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:57:15.463347 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:57:15.463361 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:57:15.463450 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0924 00:57:15.463706 (+ 256us) server_negotiation.cc:299] Negotiation successful
0924 00:57:15.463784 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":235,"threads_started":1}
I20260924 00:57:15.465628 17996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:15.452336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42547 (local address 127.8.64.129:51743)
0924 00:57:15.453304 (+ 968us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:15.453351 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:15.455753 (+ 2402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:15.456630 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:15.456641 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:15.456766 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:15.457502 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:15.457526 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:15.458635 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:15.458642 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:15.460829 (+ 2187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:15.460846 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:15.461655 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:15.461664 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:15.462025 (+ 361us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:15.462861 (+ 836us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 00:57:15.462867 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:15.462933 (+ 66us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 00:57:15.463049 (+ 116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:57:15.463472 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:57:15.463485 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:57:15.463496 (+ 11us) client_negotiation.cc:769] Sending connection context
0924 00:57:15.463688 (+ 192us) client_negotiation.cc:240] Negotiation successful
0924 00:57:15.464155 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":286,"threads_started":1}
I20260924 00:57:15.467397 17993 heartbeater.cc:347] Connected to a master server at 127.8.64.190:42547
I20260924 00:57:15.467703 17993 heartbeater.cc:464] Registering TS with master...
I20260924 00:57:15.468446 17993 heartbeater.cc:511] Master 127.8.64.190:42547 requested a full tablet report, sending...
I20260924 00:57:15.470542 17805 ts_manager.cc:194] Registered new tserver with Master: bd5dc9a6292a4f94be437b27ab9bc368 (127.8.64.129:44623)
I20260924 00:57:15.471985 17805 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.64.129:51743
W20260924 00:57:15.788889 17997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:57:15.789162 17997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:57:15.789318 17997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:57:15.796638 17997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:57:15.796741 17997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:57:15.796921 17997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.130
I20260924 00:57:15.806478 17997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:43121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:42547
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.130
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:57:15.807766 17997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:57:15.809556 17997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:57:15.822485 18002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:57:15.822513 18005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:57:15.822834 18003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:57:15.823737 17997 server_base.cc:1034] running on GCE node
I20260924 00:57:15.824401 17997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:57:15.825321 17997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:57:15.826586 17997 hybrid_clock.cc:648] HybridClock initialized: now 1790211435826525 us; error 70 us; skew 500 ppm
I20260924 00:57:15.826930 17997 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 00:57:15.829145 17997 webserver.cc:460] Webserver started at http://127.8.64.130:43079/ using document root <none> and password file <none>
I20260924 00:57:15.829851 17997 fs_manager.cc:359] Metadata directory not provided
I20260924 00:57:15.829981 17997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:57:15.830365 17997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:57:15.833097 17997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/data/instance:
uuid: "5672c0faa509486d9b7d7f2472748d63"
format_stamp: "Formatted at 2026-09-24 00:57:15 on dist-test-slave-mmhs"
server_key: "7f3d548b49cfae9af5c1a73ac08842d9"
server_key_iv: "d19c676feca45f07687108e10b507ab7"
server_key_version: "encryptionkey@0"
I20260924 00:57:15.833982 17997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance:
uuid: "5672c0faa509486d9b7d7f2472748d63"
format_stamp: "Formatted at 2026-09-24 00:57:15 on dist-test-slave-mmhs"
server_key: "7f3d548b49cfae9af5c1a73ac08842d9"
server_key_iv: "d19c676feca45f07687108e10b507ab7"
server_key_version: "encryptionkey@0"
I20260924 00:57:15.839087 17997 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260924 00:57:15.842339 18011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:57:15.843643 17997 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260924 00:57:15.843811 17997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/wal
uuid: "5672c0faa509486d9b7d7f2472748d63"
format_stamp: "Formatted at 2026-09-24 00:57:15 on dist-test-slave-mmhs"
server_key: "7f3d548b49cfae9af5c1a73ac08842d9"
server_key_iv: "d19c676feca45f07687108e10b507ab7"
server_key_version: "encryptionkey@0"
I20260924 00:57:15.844013 17997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:57:15.856827 17997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:57:15.857571 17997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:57:15.857852 17997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:57:15.858770 17997 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:57:15.860092 17997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:57:15.860210 17997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:57:15.860347 17997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:57:15.860433 17997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:57:15.893427 17997 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.130:40211
I20260924 00:57:15.893477 18124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.130:40211 every 8 connection(s)
I20260924 00:57:15.894768 17997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
I20260924 00:57:15.904822 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 17997
I20260924 00:57:15.905023 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance
I20260924 00:57:15.905584 8450 external_mini_cluster.cc:1428] Setting key 55177ea163e584b0dfeb8d10eaa268f3
I20260924 00:57:15.907116 17995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:15.897384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:59019 (local address 127.8.64.190:42547)
0924 00:57:15.897561 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:15.897566 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:15.898961 (+ 1395us) server_negotiation.cc:407] Connection header received
0924 00:57:15.900574 (+ 1613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:15.900578 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:15.900641 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:15.900728 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:15.901833 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:15.902650 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:15.904249 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:15.904646 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:15.904736 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:15.906362 (+ 1626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:57:15.906368 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:57:15.906372 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:57:15.906393 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:57:15.906434 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:57:15.906450 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:57:15.906552 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0924 00:57:15.906827 (+ 275us) server_negotiation.cc:299] Negotiation successful
0924 00:57:15.906894 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260924 00:57:15.908218 18127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:15.897882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42547 (local address 127.8.64.130:59019)
0924 00:57:15.898807 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:15.898851 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:15.900334 (+ 1483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:15.900916 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:15.900923 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:15.901004 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:15.901684 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:15.901701 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:15.902779 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:15.902783 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:15.904109 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:15.904120 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:15.904862 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:15.904870 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:15.905172 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:15.905959 (+ 787us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 00:57:15.905961 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:15.905999 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 00:57:15.906107 (+ 108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:57:15.906604 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:57:15.906616 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:57:15.906624 (+ 8us) client_negotiation.cc:769] Sending connection context
0924 00:57:15.906816 (+ 192us) client_negotiation.cc:240] Negotiation successful
0924 00:57:15.907162 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":263,"threads_started":1}
I20260924 00:57:15.908376 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.131:0
--local_ip_for_outbound_sockets=127.8.64.131
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:42547
--builtin_ntp_servers=127.8.64.148:43121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260924 00:57:15.909533 18125 heartbeater.cc:347] Connected to a master server at 127.8.64.190:42547
I20260924 00:57:15.909843 18125 heartbeater.cc:464] Registering TS with master...
I20260924 00:57:15.910771 18125 heartbeater.cc:511] Master 127.8.64.190:42547 requested a full tablet report, sending...
I20260924 00:57:15.912163 17805 ts_manager.cc:194] Registered new tserver with Master: 5672c0faa509486d9b7d7f2472748d63 (127.8.64.130:40211)
I20260924 00:57:15.912947 17805 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.64.130:59019
W20260924 00:57:16.235327 18128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:57:16.235661 18128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:57:16.235813 18128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:57:16.243158 18128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:57:16.243297 18128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:57:16.243468 18128 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.131
I20260924 00:57:16.253049 18128 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:43121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:42547
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.131
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:57:16.254487 18128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:57:16.256304 18128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:57:16.269033 18133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:57:16.268996 18136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:57:16.269071 18134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:57:16.269644 18128 server_base.cc:1034] running on GCE node
I20260924 00:57:16.270246 18128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:57:16.271226 18128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:57:16.272416 18128 hybrid_clock.cc:648] HybridClock initialized: now 1790211436272339 us; error 76 us; skew 500 ppm
I20260924 00:57:16.272704 18128 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 00:57:16.275059 18128 webserver.cc:460] Webserver started at http://127.8.64.131:41149/ using document root <none> and password file <none>
I20260924 00:57:16.275729 18128 fs_manager.cc:359] Metadata directory not provided
I20260924 00:57:16.275815 18128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:57:16.276091 18128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:57:16.278954 18128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/data/instance:
uuid: "7b553e69a94b4a77a735ccdffa52cc25"
format_stamp: "Formatted at 2026-09-24 00:57:16 on dist-test-slave-mmhs"
server_key: "a409d447c9ee5baa2ac8bb1b2111bdcc"
server_key_iv: "71316f3fa482cf96597160ceace278f1"
server_key_version: "encryptionkey@0"
I20260924 00:57:16.279714 18128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance:
uuid: "7b553e69a94b4a77a735ccdffa52cc25"
format_stamp: "Formatted at 2026-09-24 00:57:16 on dist-test-slave-mmhs"
server_key: "a409d447c9ee5baa2ac8bb1b2111bdcc"
server_key_iv: "71316f3fa482cf96597160ceace278f1"
server_key_version: "encryptionkey@0"
I20260924 00:57:16.284794 18128 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260924 00:57:16.288197 18142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:57:16.289542 18128 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260924 00:57:16.289673 18128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/wal
uuid: "7b553e69a94b4a77a735ccdffa52cc25"
format_stamp: "Formatted at 2026-09-24 00:57:16 on dist-test-slave-mmhs"
server_key: "a409d447c9ee5baa2ac8bb1b2111bdcc"
server_key_iv: "71316f3fa482cf96597160ceace278f1"
server_key_version: "encryptionkey@0"
I20260924 00:57:16.289847 18128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:57:16.318571 18128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:57:16.319397 18128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:57:16.319680 18128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:57:16.320644 18128 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:57:16.322085 18128 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:57:16.322170 18128 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:57:16.322240 18128 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:57:16.322350 18128 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:57:16.356118 18128 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.131:45949
I20260924 00:57:16.356209 18255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.131:45949 every 8 connection(s)
I20260924 00:57:16.357446 18128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
I20260924 00:57:16.363732 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 18128
I20260924 00:57:16.363925 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance
I20260924 00:57:16.364341 8450 external_mini_cluster.cc:1428] Setting key 8e23fe6de3c4718000e291310b3b97e6
I20260924 00:57:16.370551 17995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:16.359955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:44819 (local address 127.8.64.190:42547)
0924 00:57:16.360123 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:16.360127 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:16.361501 (+ 1374us) server_negotiation.cc:407] Connection header received
0924 00:57:16.363379 (+ 1878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:16.363385 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:16.363490 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:16.363617 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:16.364841 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:16.365885 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:16.367985 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:16.368364 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:16.368460 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:16.369816 (+ 1356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:57:16.369821 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:57:16.369824 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:57:16.369841 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:57:16.369869 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:57:16.369881 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:57:16.369951 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 00:57:16.370160 (+ 209us) server_negotiation.cc:299] Negotiation successful
0924 00:57:16.370230 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260924 00:57:16.371627 18258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:16.360466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42547 (local address 127.8.64.131:44819)
0924 00:57:16.361351 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:16.361390 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:16.363031 (+ 1641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:16.363838 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:16.363846 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:16.364016 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:16.364663 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:16.364682 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:16.366034 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:16.366039 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:16.367862 (+ 1823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:16.367874 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:16.368587 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:16.368592 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:16.368807 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:16.369501 (+ 694us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 00:57:16.369504 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:16.369543 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 00:57:16.369632 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:57:16.369973 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:57:16.369981 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:57:16.369986 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 00:57:16.370146 (+ 160us) client_negotiation.cc:240] Negotiation successful
0924 00:57:16.370570 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":268,"threads_started":1}
I20260924 00:57:16.373131 18256 heartbeater.cc:347] Connected to a master server at 127.8.64.190:42547
I20260924 00:57:16.373448 18256 heartbeater.cc:464] Registering TS with master...
I20260924 00:57:16.374215 18256 heartbeater.cc:511] Master 127.8.64.190:42547 requested a full tablet report, sending...
I20260924 00:57:16.375715 17805 ts_manager.cc:194] Registered new tserver with Master: 7b553e69a94b4a77a735ccdffa52cc25 (127.8.64.131:45949)
I20260924 00:57:16.376596 17805 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.64.131:44819
I20260924 00:57:16.379884 8450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260924 00:57:16.386026 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:57:16.384422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35858 (local address 127.8.64.190:42547)
0924 00:57:16.384599 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:16.384603 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:16.385141 (+ 538us) server_negotiation.cc:407] Connection header received
0924 00:57:16.385271 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:16.385274 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:16.385343 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:16.385425 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:16.385763 (+ 338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35858: BlockingRecv error: recv got EOF from 127.0.0.1:35858 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260924 00:57:16.387039 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 17864
I20260924 00:57:16.394677 18265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:16.387877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34882 (local address 127.8.64.129:44623)
0924 00:57:16.388294 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:16.388302 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:16.388326 (+ 24us) server_negotiation.cc:407] Connection header received
0924 00:57:16.388380 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:16.388386 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:16.388585 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:16.388704 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:16.389053 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:16.390028 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:16.391397 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:16.392035 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:16.392137 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:16.393178 (+ 1041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:57:16.393186 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:57:16.393200 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:57:16.393812 (+ 612us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:57:16.393857 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:57:16.393877 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:57:16.393970 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0924 00:57:16.394123 (+ 153us) server_negotiation.cc:299] Negotiation successful
0924 00:57:16.394286 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":198,"threads_started":1}
I20260924 00:57:16.395864 17988 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:57:16.477344 17993 heartbeater.cc:503] Master 127.8.64.190:42547 was elected leader, sending a full tablet report...
I20260924 00:57:16.479151 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 17864
I20260924 00:57:16.495728 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 17997
I20260924 00:57:16.503942 18267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:16.496624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35708 (local address 127.8.64.130:40211)
0924 00:57:16.497177 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:16.497184 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:16.497212 (+ 28us) server_negotiation.cc:407] Connection header received
0924 00:57:16.497277 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:16.497284 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:16.497506 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:16.497644 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:16.497965 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:16.499039 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:16.500234 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:16.500795 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:16.500886 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:16.502234 (+ 1348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:57:16.502241 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:57:16.502277 (+ 36us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:57:16.503072 (+ 795us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:57:16.503125 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:57:16.503149 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:57:16.503277 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0924 00:57:16.503399 (+ 122us) server_negotiation.cc:299] Negotiation successful
0924 00:57:16.503570 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":288,"threads_started":1}
I20260924 00:57:16.505327 18120 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:57:16.586726 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 17997
I20260924 00:57:16.603168 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 18128
I20260924 00:57:16.611676 18269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:16.604235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36602 (local address 127.8.64.131:45949)
0924 00:57:16.604957 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:16.604964 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:16.604995 (+ 31us) server_negotiation.cc:407] Connection header received
0924 00:57:16.605065 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:16.605072 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:16.605289 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:16.605415 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:16.605807 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:16.606939 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:16.608263 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:16.608864 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:16.609029 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:16.610155 (+ 1126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:57:16.610164 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:57:16.610177 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:57:16.610825 (+ 648us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:57:16.610872 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:57:16.610891 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:57:16.611035 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0924 00:57:16.611185 (+ 150us) server_negotiation.cc:299] Negotiation successful
0924 00:57:16.611315 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":416,"threads_started":1}
I20260924 00:57:16.612998 18251 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:57:16.696784 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 18128
I20260924 00:57:16.712743 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 17774
I20260924 00:57:16.714125 17835 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:57:16.814177 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 17774
2026-09-24T00:57:16Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2430 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-24T00:57:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:57:16Z Disabled control of system clock
I20260924 00:57:16.886509 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:36109
--webserver_interface=127.8.64.190
--webserver_port=0
--builtin_ntp_servers=127.8.64.148:46539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:36109
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260924 00:57:17.213754 18275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:57:17.214046 18275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:57:17.214129 18275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:57:17.221505 18275 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:57:17.221612 18275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:57:17.221652 18275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:57:17.221709 18275 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:57:17.221752 18275 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:57:17.231070 18275 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:46539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:36109
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:36109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:57:17.232323 18275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:57:17.234040 18275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:57:17.245541 18281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:57:17.245572 18280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:57:17.245635 18283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:57:17.246732 18275 server_base.cc:1034] running on GCE node
I20260924 00:57:17.247630 18275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:57:17.249130 18275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:57:17.250439 18275 hybrid_clock.cc:648] HybridClock initialized: now 1790211437250377 us; error 79 us; skew 500 ppm
I20260924 00:57:17.250808 18275 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 00:57:17.253034 18275 webserver.cc:460] Webserver started at http://127.8.64.190:36217/ using document root <none> and password file <none>
I20260924 00:57:17.253769 18275 fs_manager.cc:359] Metadata directory not provided
I20260924 00:57:17.253892 18275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:57:17.254211 18275 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:57:17.257102 18275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "9d11643d2946478ba7e2c3e8ff264ce1"
format_stamp: "Formatted at 2026-09-24 00:57:17 on dist-test-slave-mmhs"
server_key: "7d68130dc28308329792362c5bda8e13"
server_key_iv: "bf9eaffde3728997de31a58457e31134"
server_key_version: "encryptionkey@0"
I20260924 00:57:17.257942 18275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "9d11643d2946478ba7e2c3e8ff264ce1"
format_stamp: "Formatted at 2026-09-24 00:57:17 on dist-test-slave-mmhs"
server_key: "7d68130dc28308329792362c5bda8e13"
server_key_iv: "bf9eaffde3728997de31a58457e31134"
server_key_version: "encryptionkey@0"
I20260924 00:57:17.262957 18275 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260924 00:57:17.266212 18289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:57:17.267709 18275 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260924 00:57:17.267887 18275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "9d11643d2946478ba7e2c3e8ff264ce1"
format_stamp: "Formatted at 2026-09-24 00:57:17 on dist-test-slave-mmhs"
server_key: "7d68130dc28308329792362c5bda8e13"
server_key_iv: "bf9eaffde3728997de31a58457e31134"
server_key_version: "encryptionkey@0"
I20260924 00:57:17.268106 18275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:57:17.273820 18275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:57:17.274705 18275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:57:17.275004 18275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:57:17.293466 18275 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.190:36109
I20260924 00:57:17.293512 18340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.190:36109 every 8 connection(s)
I20260924 00:57:17.294867 18275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 00:57:17.298493 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 18275
I20260924 00:57:17.298722 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 00:57:17.298874 18341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:57:17.299412 8450 external_mini_cluster.cc:1428] Setting key 57423927e8a92218bdb81c0671f0a439
I20260924 00:57:17.309432 18344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:17.301498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43198 (local address 127.8.64.190:36109)
0924 00:57:17.302627 (+ 1129us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:17.302650 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:17.302705 (+ 55us) server_negotiation.cc:407] Connection header received
0924 00:57:17.305486 (+ 2781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:17.305517 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:17.306086 (+ 569us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:17.306557 (+ 471us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:17.307528 (+ 971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:57:17.307535 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:57:17.307551 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:57:17.307566 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:57:17.307638 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:57:17.307658 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:57:17.307884 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0924 00:57:17.308013 (+ 129us) server_negotiation.cc:299] Negotiation successful
0924 00:57:17.308266 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":345,"threads_started":1}
I20260924 00:57:17.313073 18341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d11643d2946478ba7e2c3e8ff264ce1: Bootstrap starting.
I20260924 00:57:17.315869 18341 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d11643d2946478ba7e2c3e8ff264ce1: Neither blocks nor log segments found. Creating new log.
I20260924 00:57:17.316854 18341 log.cc:824] T 00000000000000000000000000000000 P 9d11643d2946478ba7e2c3e8ff264ce1: Log is configured to *not* fsync() on all Append() calls
I20260924 00:57:17.319566 18341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d11643d2946478ba7e2c3e8ff264ce1: No bootstrap required, opened a new log
I20260924 00:57:17.326687 18341 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d11643d2946478ba7e2c3e8ff264ce1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d11643d2946478ba7e2c3e8ff264ce1" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 36109 } }
I20260924 00:57:17.327031 18341 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d11643d2946478ba7e2c3e8ff264ce1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:57:17.327131 18341 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d11643d2946478ba7e2c3e8ff264ce1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d11643d2946478ba7e2c3e8ff264ce1, State: Initialized, Role: FOLLOWER
I20260924 00:57:17.327684 18341 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d11643d2946478ba7e2c3e8ff264ce1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d11643d2946478ba7e2c3e8ff264ce1" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 36109 } }
I20260924 00:57:17.327922 18341 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d11643d2946478ba7e2c3e8ff264ce1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:57:17.328073 18341 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d11643d2946478ba7e2c3e8ff264ce1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:57:17.328274 18341 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d11643d2946478ba7e2c3e8ff264ce1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:57:17.330526 18341 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d11643d2946478ba7e2c3e8ff264ce1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d11643d2946478ba7e2c3e8ff264ce1" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 36109 } }
I20260924 00:57:17.331010 18341 leader_election.cc:302] T 00000000000000000000000000000000 P 9d11643d2946478ba7e2c3e8ff264ce1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d11643d2946478ba7e2c3e8ff264ce1; no voters:
I20260924 00:57:17.331614 18341 leader_election.cc:288] T 00000000000000000000000000000000 P 9d11643d2946478ba7e2c3e8ff264ce1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:57:17.331745 18346 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d11643d2946478ba7e2c3e8ff264ce1 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:57:17.332455 18346 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d11643d2946478ba7e2c3e8ff264ce1 [term 1 LEADER]: Becoming Leader. State: Replica: 9d11643d2946478ba7e2c3e8ff264ce1, State: Running, Role: LEADER
I20260924 00:57:17.333011 18346 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d11643d2946478ba7e2c3e8ff264ce1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d11643d2946478ba7e2c3e8ff264ce1" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 36109 } }
I20260924 00:57:17.333725 18341 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d11643d2946478ba7e2c3e8ff264ce1 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:57:17.336032 18348 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d11643d2946478ba7e2c3e8ff264ce1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d11643d2946478ba7e2c3e8ff264ce1. Latest consensus state: current_term: 1 leader_uuid: "9d11643d2946478ba7e2c3e8ff264ce1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d11643d2946478ba7e2c3e8ff264ce1" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 36109 } } }
I20260924 00:57:17.336481 18348 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d11643d2946478ba7e2c3e8ff264ce1 [sys.catalog]: This master's current role is: LEADER
I20260924 00:57:17.337138 18347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d11643d2946478ba7e2c3e8ff264ce1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d11643d2946478ba7e2c3e8ff264ce1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d11643d2946478ba7e2c3e8ff264ce1" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 36109 } } }
I20260924 00:57:17.337519 18347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d11643d2946478ba7e2c3e8ff264ce1 [sys.catalog]: This master's current role is: LEADER
I20260924 00:57:17.339370 18353 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:57:17.343850 18353 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:57:17.352803 18353 catalog_manager.cc:1347] Generated new cluster ID: 13b6779eeae3406baaab3742d6c9b8f1
I20260924 00:57:17.352967 18353 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:57:17.368892 18353 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:57:17.369325 18353 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:57:17.378749 18353 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d11643d2946478ba7e2c3e8ff264ce1: Generated new TSK 0
I20260924 00:57:17.379602 18353 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 00:57:17.389393 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.129:0
--local_ip_for_outbound_sockets=127.8.64.129
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:36109
--builtin_ntp_servers=127.8.64.148:46539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260924 00:57:17.726176 18365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:57:17.726492 18365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:57:17.726583 18365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:57:17.733876 18365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:57:17.733984 18365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:57:17.734094 18365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.129
I20260924 00:57:17.743455 18365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:46539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:36109
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.129
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:57:17.744796 18365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:57:17.746564 18365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:57:17.759665 18373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:57:17.759745 18371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:57:17.759745 18370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:57:17.760679 18365 server_base.cc:1034] running on GCE node
I20260924 00:57:17.761256 18365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:57:17.762210 18365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:57:17.763407 18365 hybrid_clock.cc:648] HybridClock initialized: now 1790211437763366 us; error 39 us; skew 500 ppm
I20260924 00:57:17.763725 18365 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 00:57:17.765859 18365 webserver.cc:460] Webserver started at http://127.8.64.129:43859/ using document root <none> and password file <none>
I20260924 00:57:17.766609 18365 fs_manager.cc:359] Metadata directory not provided
I20260924 00:57:17.766722 18365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:57:17.767166 18365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:57:17.773138 18365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/data/instance:
uuid: "63d91129ddc84624a34ee577c4e6334e"
format_stamp: "Formatted at 2026-09-24 00:57:17 on dist-test-slave-mmhs"
server_key: "66f83a749cc4c12d44241e1051764917"
server_key_iv: "6d073ee500ea3ff0c2afb031775eb347"
server_key_version: "encryptionkey@0"
I20260924 00:57:17.774181 18365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance:
uuid: "63d91129ddc84624a34ee577c4e6334e"
format_stamp: "Formatted at 2026-09-24 00:57:17 on dist-test-slave-mmhs"
server_key: "66f83a749cc4c12d44241e1051764917"
server_key_iv: "6d073ee500ea3ff0c2afb031775eb347"
server_key_version: "encryptionkey@0"
I20260924 00:57:17.779850 18365 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260924 00:57:17.783352 18379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:57:17.784883 18365 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260924 00:57:17.785076 18365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/wal
uuid: "63d91129ddc84624a34ee577c4e6334e"
format_stamp: "Formatted at 2026-09-24 00:57:17 on dist-test-slave-mmhs"
server_key: "66f83a749cc4c12d44241e1051764917"
server_key_iv: "6d073ee500ea3ff0c2afb031775eb347"
server_key_version: "encryptionkey@0"
I20260924 00:57:17.785332 18365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:57:17.791370 18365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:57:17.792194 18365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:57:17.792448 18365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:57:17.793370 18365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:57:17.795101 18365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:57:17.795203 18365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:57:17.795284 18365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:57:17.795338 18365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:57:17.828878 18365 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.129:42663
I20260924 00:57:17.829016 18491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.129:42663 every 8 connection(s)
I20260924 00:57:17.830224 18365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
I20260924 00:57:17.833175 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 18365
I20260924 00:57:17.833407 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance
I20260924 00:57:17.834259 8450 external_mini_cluster.cc:1428] Setting key 4cd2105eb6eeeb076e0e343a7b5c633d
I20260924 00:57:17.837513 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.130:0
--local_ip_for_outbound_sockets=127.8.64.130
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:36109
--builtin_ntp_servers=127.8.64.148:46539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260924 00:57:17.840101 18494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:17.832889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:44511 (local address 127.8.64.190:36109)
0924 00:57:17.833410 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:17.833417 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:17.834999 (+ 1582us) server_negotiation.cc:407] Connection header received
0924 00:57:17.837265 (+ 2266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:17.837269 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:17.837319 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:17.837448 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:17.839341 (+ 1893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:57:17.839345 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:57:17.839348 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:57:17.839393 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:57:17.839434 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:57:17.839447 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:57:17.839548 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0924 00:57:17.839825 (+ 277us) server_negotiation.cc:299] Negotiation successful
0924 00:57:17.839888 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":261,"threads_started":1}
I20260924 00:57:17.841476 18495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:17.833474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:36109 (local address 127.8.64.129:44511)
0924 00:57:17.834774 (+ 1300us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:17.834831 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:17.836921 (+ 2090us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:17.837986 (+ 1065us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:17.838002 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:17.838172 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:17.838955 (+ 783us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 00:57:17.838961 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:17.839019 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 00:57:17.839144 (+ 125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:57:17.839562 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:57:17.839577 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:57:17.839589 (+ 12us) client_negotiation.cc:769] Sending connection context
0924 00:57:17.839896 (+ 307us) client_negotiation.cc:240] Negotiation successful
0924 00:57:17.840166 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":352,"threads_started":1}
I20260924 00:57:17.843250 18492 heartbeater.cc:347] Connected to a master server at 127.8.64.190:36109
I20260924 00:57:17.843608 18492 heartbeater.cc:464] Registering TS with master...
I20260924 00:57:17.844364 18492 heartbeater.cc:511] Master 127.8.64.190:36109 requested a full tablet report, sending...
I20260924 00:57:17.846884 18306 ts_manager.cc:194] Registered new tserver with Master: 63d91129ddc84624a34ee577c4e6334e (127.8.64.129:42663)
W20260924 00:57:18.169298 18496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:57:18.169591 18496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:57:18.169682 18496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:57:18.177246 18496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:57:18.177363 18496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:57:18.177475 18496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.130
I20260924 00:57:18.186676 18496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:46539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:36109
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.130
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:57:18.187896 18496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:57:18.190403 18496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:57:18.208389 18502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:57:18.208935 18504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:57:18.209182 18501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:57:18.210660 18496 server_base.cc:1034] running on GCE node
I20260924 00:57:18.211490 18496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:57:18.212852 18496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:57:18.214147 18496 hybrid_clock.cc:648] HybridClock initialized: now 1790211438214066 us; error 67 us; skew 500 ppm
I20260924 00:57:18.214593 18496 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 00:57:18.217453 18496 webserver.cc:460] Webserver started at http://127.8.64.130:39483/ using document root <none> and password file <none>
I20260924 00:57:18.218120 18496 fs_manager.cc:359] Metadata directory not provided
I20260924 00:57:18.218209 18496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:57:18.218569 18496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:57:18.221372 18496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/data/instance:
uuid: "2f75cbd386764d0a8be689b043d438fc"
format_stamp: "Formatted at 2026-09-24 00:57:18 on dist-test-slave-mmhs"
server_key: "02f425a34daffb725916f56f7e9956bb"
server_key_iv: "536822a5b2cefe4e1b1ea833341ae0a9"
server_key_version: "encryptionkey@0"
I20260924 00:57:18.222115 18496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance:
uuid: "2f75cbd386764d0a8be689b043d438fc"
format_stamp: "Formatted at 2026-09-24 00:57:18 on dist-test-slave-mmhs"
server_key: "02f425a34daffb725916f56f7e9956bb"
server_key_iv: "536822a5b2cefe4e1b1ea833341ae0a9"
server_key_version: "encryptionkey@0"
I20260924 00:57:18.227417 18496 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260924 00:57:18.230816 18510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:57:18.232167 18496 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260924 00:57:18.232344 18496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/wal
uuid: "2f75cbd386764d0a8be689b043d438fc"
format_stamp: "Formatted at 2026-09-24 00:57:18 on dist-test-slave-mmhs"
server_key: "02f425a34daffb725916f56f7e9956bb"
server_key_iv: "536822a5b2cefe4e1b1ea833341ae0a9"
server_key_version: "encryptionkey@0"
I20260924 00:57:18.232579 18496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:57:18.238488 18496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:57:18.239279 18496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:57:18.239552 18496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:57:18.240432 18496 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:57:18.241860 18496 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:57:18.241981 18496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:57:18.242128 18496 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:57:18.242216 18496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:57:18.275892 18496 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.130:40155
I20260924 00:57:18.275944 18622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.130:40155 every 8 connection(s)
I20260924 00:57:18.277248 18496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
I20260924 00:57:18.284616 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 18496
I20260924 00:57:18.284825 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance
I20260924 00:57:18.285251 8450 external_mini_cluster.cc:1428] Setting key 28de0f896785d158733cdf4554b37c91
I20260924 00:57:18.286062 18494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:18.279901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:58835 (local address 127.8.64.190:36109)
0924 00:57:18.280074 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:18.280079 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:18.281642 (+ 1563us) server_negotiation.cc:407] Connection header received
0924 00:57:18.283582 (+ 1940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:18.283588 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:18.283683 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:18.283805 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:18.285328 (+ 1523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:57:18.285336 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:57:18.285343 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:57:18.285366 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:57:18.285408 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:57:18.285424 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:57:18.285558 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0924 00:57:18.285736 (+ 178us) server_negotiation.cc:299] Negotiation successful
0924 00:57:18.285802 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260924 00:57:18.287467 18625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:18.280449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:36109 (local address 127.8.64.130:58835)
0924 00:57:18.281458 (+ 1009us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:18.281503 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:18.283285 (+ 1782us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:18.284059 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:18.284069 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:18.284172 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:18.285002 (+ 830us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 00:57:18.285004 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:18.285045 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 00:57:18.285135 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:57:18.285530 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:57:18.285538 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:57:18.285544 (+ 6us) client_negotiation.cc:769] Sending connection context
0924 00:57:18.285776 (+ 232us) client_negotiation.cc:240] Negotiation successful
0924 00:57:18.286060 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":318,"threads_started":1}
I20260924 00:57:18.288417 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.131:0
--local_ip_for_outbound_sockets=127.8.64.131
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:36109
--builtin_ntp_servers=127.8.64.148:46539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260924 00:57:18.288913 18623 heartbeater.cc:347] Connected to a master server at 127.8.64.190:36109
I20260924 00:57:18.289235 18623 heartbeater.cc:464] Registering TS with master...
I20260924 00:57:18.289880 18623 heartbeater.cc:511] Master 127.8.64.190:36109 requested a full tablet report, sending...
I20260924 00:57:18.291314 18306 ts_manager.cc:194] Registered new tserver with Master: 2f75cbd386764d0a8be689b043d438fc (127.8.64.130:40155)
W20260924 00:57:18.621575 18626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:57:18.621846 18626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:57:18.621994 18626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:57:18.629109 18626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:57:18.629212 18626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:57:18.629446 18626 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.131
I20260924 00:57:18.641595 18626 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:46539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:36109
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.131
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:57:18.643067 18626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:57:18.644877 18626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:57:18.657866 18632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:57:18.657910 18631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:57:18.657912 18634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:57:18.658720 18626 server_base.cc:1034] running on GCE node
I20260924 00:57:18.659248 18626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:57:18.660135 18626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:57:18.661351 18626 hybrid_clock.cc:648] HybridClock initialized: now 1790211438661278 us; error 63 us; skew 500 ppm
I20260924 00:57:18.661685 18626 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 00:57:18.663968 18626 webserver.cc:460] Webserver started at http://127.8.64.131:37451/ using document root <none> and password file <none>
I20260924 00:57:18.664688 18626 fs_manager.cc:359] Metadata directory not provided
I20260924 00:57:18.664798 18626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:57:18.665135 18626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:57:18.668006 18626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/data/instance:
uuid: "2b008f7a432c4e8299fc26b61bfe4613"
format_stamp: "Formatted at 2026-09-24 00:57:18 on dist-test-slave-mmhs"
server_key: "a246b7eea6ac36c9067eb9b957acbba5"
server_key_iv: "e6b9ecbdaeec4cb990e0fab706b2003d"
server_key_version: "encryptionkey@0"
I20260924 00:57:18.668798 18626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance:
uuid: "2b008f7a432c4e8299fc26b61bfe4613"
format_stamp: "Formatted at 2026-09-24 00:57:18 on dist-test-slave-mmhs"
server_key: "a246b7eea6ac36c9067eb9b957acbba5"
server_key_iv: "e6b9ecbdaeec4cb990e0fab706b2003d"
server_key_version: "encryptionkey@0"
I20260924 00:57:18.674077 18626 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260924 00:57:18.677789 18640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:57:18.679472 18626 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 00:57:18.679662 18626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/wal
uuid: "2b008f7a432c4e8299fc26b61bfe4613"
format_stamp: "Formatted at 2026-09-24 00:57:18 on dist-test-slave-mmhs"
server_key: "a246b7eea6ac36c9067eb9b957acbba5"
server_key_iv: "e6b9ecbdaeec4cb990e0fab706b2003d"
server_key_version: "encryptionkey@0"
I20260924 00:57:18.679910 18626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:57:18.685842 18626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:57:18.686655 18626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:57:18.686960 18626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:57:18.687971 18626 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:57:18.689476 18626 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:57:18.689594 18626 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:57:18.689720 18626 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:57:18.689812 18626 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:57:18.724030 18626 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.131:35295
I20260924 00:57:18.724086 18752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.131:35295 every 8 connection(s)
I20260924 00:57:18.725327 18626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
I20260924 00:57:18.732863 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 18626
I20260924 00:57:18.733075 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance
I20260924 00:57:18.733502 18494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:18.727913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:54339 (local address 127.8.64.190:36109)
0924 00:57:18.728091 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:18.728096 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:18.729486 (+ 1390us) server_negotiation.cc:407] Connection header received
0924 00:57:18.731274 (+ 1788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:18.731279 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:18.731349 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:18.731455 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:18.732704 (+ 1249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:57:18.732710 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:57:18.732715 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:57:18.732733 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:57:18.732769 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:57:18.732780 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:57:18.732907 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0924 00:57:18.733160 (+ 253us) server_negotiation.cc:299] Negotiation successful
0924 00:57:18.733226 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260924 00:57:18.733651 8450 external_mini_cluster.cc:1428] Setting key 886c9dc48c861ce32c5493937d86918f
I20260924 00:57:18.734772 18755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:18.728442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:36109 (local address 127.8.64.131:54339)
0924 00:57:18.729331 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:18.729367 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:18.730993 (+ 1626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:18.731677 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:18.731685 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:18.731776 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:18.732403 (+ 627us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 00:57:18.732405 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:18.732440 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 00:57:18.732542 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:57:18.732869 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:57:18.732878 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:57:18.732887 (+ 9us) client_negotiation.cc:769] Sending connection context
0924 00:57:18.733229 (+ 342us) client_negotiation.cc:240] Negotiation successful
0924 00:57:18.733568 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":263,"threads_started":1}
I20260924 00:57:18.736176 18753 heartbeater.cc:347] Connected to a master server at 127.8.64.190:36109
I20260924 00:57:18.736510 18753 heartbeater.cc:464] Registering TS with master...
I20260924 00:57:18.737088 18753 heartbeater.cc:511] Master 127.8.64.190:36109 requested a full tablet report, sending...
I20260924 00:57:18.738420 18306 ts_manager.cc:194] Registered new tserver with Master: 2b008f7a432c4e8299fc26b61bfe4613 (127.8.64.131:35295)
I20260924 00:57:18.748903 8450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260924 00:57:18.755120 18494 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:57:18.753400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43212 (local address 127.8.64.190:36109)
0924 00:57:18.753555 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:18.753560 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:18.754112 (+ 552us) server_negotiation.cc:407] Connection header received
0924 00:57:18.754369 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:18.754374 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:18.754435 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:18.754547 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:18.754893 (+ 346us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43212: BlockingRecv error: recv got EOF from 127.0.0.1:43212 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260924 00:57:18.755592 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 18365
I20260924 00:57:18.759173 18762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:18.756706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37018 (local address 127.8.64.129:42663)
0924 00:57:18.757203 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:18.757210 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:18.757239 (+ 29us) server_negotiation.cc:407] Connection header received
0924 00:57:18.757337 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:18.757345 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:18.757568 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:18.757709 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:18.758559 (+ 850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:57:18.758565 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:57:18.758577 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:57:18.758591 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:57:18.758635 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:57:18.758656 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:57:18.758773 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0924 00:57:18.758846 (+ 73us) server_negotiation.cc:299] Negotiation successful
0924 00:57:18.758934 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":232,"threads_started":1}
I20260924 00:57:18.760471 18487 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:57:18.841284 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 18365
I20260924 00:57:18.857736 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 18496
I20260924 00:57:18.861027 18764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:18.858663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54770 (local address 127.8.64.130:40155)
0924 00:57:18.859182 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:18.859189 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:18.859216 (+ 27us) server_negotiation.cc:407] Connection header received
0924 00:57:18.859278 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:18.859284 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:18.859499 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:18.859625 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:18.860448 (+ 823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:57:18.860455 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:57:18.860465 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:57:18.860476 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:57:18.860517 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:57:18.860533 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:57:18.860639 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0924 00:57:18.860712 (+ 73us) server_negotiation.cc:299] Negotiation successful
0924 00:57:18.860795 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":245,"threads_started":1}
I20260924 00:57:18.862195 18618 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:57:18.941650 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 18496
I20260924 00:57:18.958889 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 18626
I20260924 00:57:18.962556 18766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:18.959901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58148 (local address 127.8.64.131:35295)
0924 00:57:18.960531 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:18.960556 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:18.960587 (+ 31us) server_negotiation.cc:407] Connection header received
0924 00:57:18.960652 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:18.960659 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:18.960878 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:18.961004 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:18.961875 (+ 871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:57:18.961881 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:57:18.961892 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:57:18.961907 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:57:18.961944 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:57:18.961958 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:57:18.962089 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0924 00:57:18.962138 (+ 49us) server_negotiation.cc:299] Negotiation successful
0924 00:57:18.962213 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":492,"thread_start_us":335,"threads_started":1}
I20260924 00:57:18.963776 18748 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:57:19.044750 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 18626
I20260924 00:57:19.061178 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 18275
I20260924 00:57:19.062428 18336 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:57:19.161549 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 18275
2026-09-24T00:57:19Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2347 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:57:19 dist-test-slave-mmhs krb5kdc[18771](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:57:19 dist-test-slave-mmhs krb5kdc[18771](info): set up 2 sockets
Sep 24 00:57:19 dist-test-slave-mmhs krb5kdc[18771](info): commencing operation
krb5kdc: starting...
W20260924 00:57:20.268889 8450 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.002s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Sep 24 00:57:20 dist-test-slave-mmhs krb5kdc[18771](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211440, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:57:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:57:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.64.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
I20260924 00:57:20.510440 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:33751
--webserver_interface=127.8.64.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:43283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:33751
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:57:20.835086 18784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:57:20.835469 18784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:57:20.835635 18784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:57:20.842767 18784 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:57:20.842918 18784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:57:20.843029 18784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:57:20.843138 18784 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:57:20.843225 18784 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:57:20.852926 18784 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:43283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:33751
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:33751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:57:20.854247 18784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:57:20.855585 18784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:57:20.867517 18792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:57:20.867635 18789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:57:20.867633 18790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:57:20.868700 18784 server_base.cc:1034] running on GCE node
I20260924 00:57:20.869552 18784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:57:20.871042 18784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:57:20.872246 18784 hybrid_clock.cc:648] HybridClock initialized: now 1790211440872206 us; error 33 us; skew 500 ppm
I20260924 00:57:20.872593 18784 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:57:20 dist-test-slave-mmhs krb5kdc[18771](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211440, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:57:20.880177 18784 init.cc:377] Logged in from keytab as kudu/127.8.64.190@KRBTEST.COM (short username kudu)
I20260924 00:57:20.881814 18784 webserver.cc:460] Webserver started at http://127.8.64.190:41733/ using document root <none> and password file <none>
I20260924 00:57:20.882612 18784 fs_manager.cc:359] Metadata directory not provided
I20260924 00:57:20.882763 18784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:57:20.883070 18784 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:57:20.886045 18784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "a1b932c316da4d549f0c193e9155fe2e"
format_stamp: "Formatted at 2026-09-24 00:57:20 on dist-test-slave-mmhs"
server_key: "6bd6e08fd85fd6aa407d85320ac2ccc6"
server_key_iv: "4b630ed3129b9c5481b6f4d219bca7b1"
server_key_version: "encryptionkey@0"
I20260924 00:57:20.886868 18784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "a1b932c316da4d549f0c193e9155fe2e"
format_stamp: "Formatted at 2026-09-24 00:57:20 on dist-test-slave-mmhs"
server_key: "6bd6e08fd85fd6aa407d85320ac2ccc6"
server_key_iv: "4b630ed3129b9c5481b6f4d219bca7b1"
server_key_version: "encryptionkey@0"
I20260924 00:57:20.891976 18784 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260924 00:57:20.895447 18799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:57:20.896817 18784 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260924 00:57:20.897020 18784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "a1b932c316da4d549f0c193e9155fe2e"
format_stamp: "Formatted at 2026-09-24 00:57:20 on dist-test-slave-mmhs"
server_key: "6bd6e08fd85fd6aa407d85320ac2ccc6"
server_key_iv: "4b630ed3129b9c5481b6f4d219bca7b1"
server_key_version: "encryptionkey@0"
I20260924 00:57:20.897228 18784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:57:20.915104 18784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:57:20.919299 18784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:57:20.919591 18784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:57:20.939102 18784 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.190:33751
I20260924 00:57:20.939122 18850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.190:33751 every 8 connection(s)
I20260924 00:57:20.940543 18784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 00:57:20.943441 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 18784
I20260924 00:57:20.943688 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 00:57:20.944473 8450 external_mini_cluster.cc:1428] Setting key 41fccaa5f275fc806a57af1820e8e6ec
I20260924 00:57:20.944665 18851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:57:20.959039 18851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1b932c316da4d549f0c193e9155fe2e: Bootstrap starting.
Sep 24 00:57:20 dist-test-slave-mmhs krb5kdc[18771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211440, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:57:20.962996 18851 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1b932c316da4d549f0c193e9155fe2e: Neither blocks nor log segments found. Creating new log.
I20260924 00:57:20.964339 18851 log.cc:824] T 00000000000000000000000000000000 P a1b932c316da4d549f0c193e9155fe2e: Log is configured to *not* fsync() on all Append() calls
I20260924 00:57:20.967738 18851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1b932c316da4d549f0c193e9155fe2e: No bootstrap required, opened a new log
I20260924 00:57:20.968966 18854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:20.946594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50614 (local address 127.8.64.190:33751)
0924 00:57:20.947402 (+ 808us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:20.947416 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:20.947456 (+ 40us) server_negotiation.cc:407] Connection header received
0924 00:57:20.949192 (+ 1736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:20.949213 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:20.949613 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:20.949936 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:20.951739 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:20.953004 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:20.954822 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:20.955496 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:20.955639 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:20.960321 (+ 4682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:57:20.960331 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:57:20.960350 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:57:20.960403 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:57:20.964437 (+ 4034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:57:20.965336 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:57:20.965342 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:57:20.965348 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:57:20.965532 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:57:20.965912 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:57:20.965917 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:57:20.965920 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:57:20.966605 (+ 685us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:57:20.966901 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0924 00:57:20.967229 (+ 328us) server_negotiation.cc:299] Negotiation successful
0924 00:57:20.967633 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":360,"threads_started":1}
I20260924 00:57:20.975914 18851 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1b932c316da4d549f0c193e9155fe2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1b932c316da4d549f0c193e9155fe2e" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 33751 } }
I20260924 00:57:20.976272 18851 raft_consensus.cc:374] T 00000000000000000000000000000000 P a1b932c316da4d549f0c193e9155fe2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:57:20.976351 18851 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1b932c316da4d549f0c193e9155fe2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1b932c316da4d549f0c193e9155fe2e, State: Initialized, Role: FOLLOWER
I20260924 00:57:20.976886 18851 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1b932c316da4d549f0c193e9155fe2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1b932c316da4d549f0c193e9155fe2e" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 33751 } }
I20260924 00:57:20.977078 18851 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1b932c316da4d549f0c193e9155fe2e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:57:20.977175 18851 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1b932c316da4d549f0c193e9155fe2e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:57:20.977324 18851 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1b932c316da4d549f0c193e9155fe2e [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:57:20.979741 18851 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1b932c316da4d549f0c193e9155fe2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1b932c316da4d549f0c193e9155fe2e" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 33751 } }
I20260924 00:57:20.980239 18851 leader_election.cc:302] T 00000000000000000000000000000000 P a1b932c316da4d549f0c193e9155fe2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1b932c316da4d549f0c193e9155fe2e; no voters:
I20260924 00:57:20.980823 18851 leader_election.cc:288] T 00000000000000000000000000000000 P a1b932c316da4d549f0c193e9155fe2e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:57:20.980983 18856 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1b932c316da4d549f0c193e9155fe2e [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:57:20.982013 18856 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1b932c316da4d549f0c193e9155fe2e [term 1 LEADER]: Becoming Leader. State: Replica: a1b932c316da4d549f0c193e9155fe2e, State: Running, Role: LEADER
I20260924 00:57:20.982616 18856 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1b932c316da4d549f0c193e9155fe2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1b932c316da4d549f0c193e9155fe2e" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 33751 } }
I20260924 00:57:20.983167 18851 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1b932c316da4d549f0c193e9155fe2e [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:57:20.985412 18857 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1b932c316da4d549f0c193e9155fe2e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a1b932c316da4d549f0c193e9155fe2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1b932c316da4d549f0c193e9155fe2e" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 33751 } } }
I20260924 00:57:20.985733 18857 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1b932c316da4d549f0c193e9155fe2e [sys.catalog]: This master's current role is: LEADER
I20260924 00:57:20.985523 18858 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1b932c316da4d549f0c193e9155fe2e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1b932c316da4d549f0c193e9155fe2e. Latest consensus state: current_term: 1 leader_uuid: "a1b932c316da4d549f0c193e9155fe2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1b932c316da4d549f0c193e9155fe2e" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 33751 } } }
I20260924 00:57:20.985924 18858 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1b932c316da4d549f0c193e9155fe2e [sys.catalog]: This master's current role is: LEADER
I20260924 00:57:20.987565 18863 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:57:20.992026 18863 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:57:21.002823 18863 catalog_manager.cc:1347] Generated new cluster ID: 3fc645a886e249b4bc751f7df088ac1c
I20260924 00:57:21.002951 18863 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:57:21.028908 18863 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:57:21.030205 18863 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:57:21.044605 18863 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a1b932c316da4d549f0c193e9155fe2e: Generated new TSK 0
I20260924 00:57:21.045611 18863 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.64.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab.
Entry for principal kudu/127.8.64.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab.
I20260924 00:57:21.110524 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.129:0
--local_ip_for_outbound_sockets=127.8.64.129
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:33751
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab
--principal=kudu/127.8.64.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:43283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:57:21.439822 18877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:57:21.440099 18877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:57:21.440191 18877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:57:21.447177 18877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:57:21.447284 18877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:57:21.447393 18877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.129
I20260924 00:57:21.456698 18877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:43283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab
--principal=kudu/127.8.64.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:33751
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.129
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:57:21.458024 18877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:57:21.459290 18877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:57:21.472358 18883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:57:21.472424 18882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:57:21.472313 18885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:57:21.473857 18877 server_base.cc:1034] running on GCE node
I20260924 00:57:21.474509 18877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:57:21.475446 18877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:57:21.476673 18877 hybrid_clock.cc:648] HybridClock initialized: now 1790211441476637 us; error 46 us; skew 500 ppm
I20260924 00:57:21.476982 18877 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:57:21 dist-test-slave-mmhs krb5kdc[18771](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211441, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:57:21.480087 18877 init.cc:377] Logged in from keytab as kudu/127.8.64.129@KRBTEST.COM (short username kudu)
I20260924 00:57:21.481462 18877 webserver.cc:460] Webserver started at http://127.8.64.129:39525/ using document root <none> and password file <none>
I20260924 00:57:21.482161 18877 fs_manager.cc:359] Metadata directory not provided
I20260924 00:57:21.482352 18877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:57:21.482786 18877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:57:21.485657 18877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/data/instance:
uuid: "14258d929a9d4993846af747a2ba1d7c"
format_stamp: "Formatted at 2026-09-24 00:57:21 on dist-test-slave-mmhs"
server_key: "865c2e425585ee34d2844bdbd88ef64a"
server_key_iv: "e736540dc9d05c794eeaa3f7902afc76"
server_key_version: "encryptionkey@0"
I20260924 00:57:21.486497 18877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance:
uuid: "14258d929a9d4993846af747a2ba1d7c"
format_stamp: "Formatted at 2026-09-24 00:57:21 on dist-test-slave-mmhs"
server_key: "865c2e425585ee34d2844bdbd88ef64a"
server_key_iv: "e736540dc9d05c794eeaa3f7902afc76"
server_key_version: "encryptionkey@0"
I20260924 00:57:21.491657 18877 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260924 00:57:21.495326 18892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:57:21.496731 18877 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260924 00:57:21.496910 18877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/wal
uuid: "14258d929a9d4993846af747a2ba1d7c"
format_stamp: "Formatted at 2026-09-24 00:57:21 on dist-test-slave-mmhs"
server_key: "865c2e425585ee34d2844bdbd88ef64a"
server_key_iv: "e736540dc9d05c794eeaa3f7902afc76"
server_key_version: "encryptionkey@0"
I20260924 00:57:21.497109 18877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:57:21.528720 18877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:57:21.532745 18877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:57:21.533056 18877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:57:21.534029 18877 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:57:21.535508 18877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:57:21.535610 18877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:57:21.535768 18877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:57:21.535841 18877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:57:21.568993 18877 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.129:45401
I20260924 00:57:21.569051 19004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.129:45401 every 8 connection(s)
I20260924 00:57:21.570360 18877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
I20260924 00:57:21.574764 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 18877
I20260924 00:57:21.575002 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance
I20260924 00:57:21.575822 8450 external_mini_cluster.cc:1428] Setting key ac7604687fafc41ef8ae61f1f2a4dc60
Sep 24 00:57:21 dist-test-slave-mmhs krb5kdc[18771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211441, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:57:21.592903 19007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:21.573003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:45529 (local address 127.8.64.190:33751)
0924 00:57:21.573430 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:21.573434 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:21.574573 (+ 1139us) server_negotiation.cc:407] Connection header received
0924 00:57:21.577140 (+ 2567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:21.577146 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:21.577219 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:21.577334 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:21.579356 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:21.580078 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:21.582569 (+ 2491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:21.582959 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:21.583067 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:21.587282 (+ 4215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:57:21.587287 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:57:21.587290 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:57:21.587321 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:57:21.589802 (+ 2481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:57:21.590595 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:57:21.590608 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:57:21.590611 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:57:21.590721 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:57:21.591151 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:57:21.591155 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:57:21.591159 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:57:21.591404 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:57:21.591576 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0924 00:57:21.592467 (+ 891us) server_negotiation.cc:299] Negotiation successful
0924 00:57:21.592635 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":324,"thread_start_us":226,"threads_started":1}
I20260924 00:57:21.594601 19008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:21.573634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:33751 (local address 127.8.64.129:45529)
0924 00:57:21.574379 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:21.574439 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:21.576820 (+ 2381us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:21.577567 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:21.577576 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:21.578300 (+ 724us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:21.579163 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:21.579192 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:21.580253 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:21.580260 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:21.582428 (+ 2168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:21.582445 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:21.583200 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:21.583209 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:21.583585 (+ 376us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:21.584322 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:57:21.584356 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:21.587047 (+ 2691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:57:21.589943 (+ 2896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:57:21.589952 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:57:21.589969 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:57:21.590463 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:57:21.590829 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:57:21.590835 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:57:21.590840 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:57:21.591031 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:57:21.591586 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:57:21.591597 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:57:21.592071 (+ 474us) client_negotiation.cc:769] Sending connection context
0924 00:57:21.592388 (+ 317us) client_negotiation.cc:240] Negotiation successful
0924 00:57:21.592999 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":274,"threads_started":1}
I20260924 00:57:21.596505 19005 heartbeater.cc:347] Connected to a master server at 127.8.64.190:33751
I20260924 00:57:21.596844 19005 heartbeater.cc:464] Registering TS with master...
I20260924 00:57:21.597638 19005 heartbeater.cc:511] Master 127.8.64.190:33751 requested a full tablet report, sending...
I20260924 00:57:21.599885 18816 ts_manager.cc:194] Registered new tserver with Master: 14258d929a9d4993846af747a2ba1d7c (127.8.64.129:45401)
WARNING: no policy specified for kudu/127.8.64.130@KRBTEST.COM; defaulting to no policy
I20260924 00:57:21.601894 18816 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.129@KRBTEST.COM'} at 127.8.64.129:45529
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab.
Entry for principal kudu/127.8.64.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab.
I20260924 00:57:21.633971 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.130:0
--local_ip_for_outbound_sockets=127.8.64.130
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:33751
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab
--principal=kudu/127.8.64.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:43283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:57:21.961592 19011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:57:21.961861 19011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:57:21.962060 19011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:57:21.969345 19011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:57:21.969486 19011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:57:21.969692 19011 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.130
I20260924 00:57:21.979378 19011 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:43283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab
--principal=kudu/127.8.64.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:33751
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.130
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:57:21.980672 19011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:57:21.982002 19011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:57:21.995527 19017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:57:21.995779 19016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:57:21.996132 19019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:57:21.996824 19011 server_base.cc:1034] running on GCE node
I20260924 00:57:21.997365 19011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:57:21.998247 19011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:57:21.999521 19011 hybrid_clock.cc:648] HybridClock initialized: now 1790211441999466 us; error 45 us; skew 500 ppm
I20260924 00:57:21.999837 19011 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:57:22 dist-test-slave-mmhs krb5kdc[18771](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211442, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:57:22.003134 19011 init.cc:377] Logged in from keytab as kudu/127.8.64.130@KRBTEST.COM (short username kudu)
I20260924 00:57:22.004505 19011 webserver.cc:460] Webserver started at http://127.8.64.130:35485/ using document root <none> and password file <none>
I20260924 00:57:22.005112 19011 fs_manager.cc:359] Metadata directory not provided
I20260924 00:57:22.005213 19011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:57:22.005452 19011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:57:22.008279 19011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/data/instance:
uuid: "13e4c7a19c9a4311bda3be760a64a57f"
format_stamp: "Formatted at 2026-09-24 00:57:22 on dist-test-slave-mmhs"
server_key: "f2c75f0f9c67bd8e56bcba369ce68346"
server_key_iv: "7325124fdf5b925e4530a76c11ca15f9"
server_key_version: "encryptionkey@0"
I20260924 00:57:22.008998 19011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance:
uuid: "13e4c7a19c9a4311bda3be760a64a57f"
format_stamp: "Formatted at 2026-09-24 00:57:22 on dist-test-slave-mmhs"
server_key: "f2c75f0f9c67bd8e56bcba369ce68346"
server_key_iv: "7325124fdf5b925e4530a76c11ca15f9"
server_key_version: "encryptionkey@0"
I20260924 00:57:22.014528 19011 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260924 00:57:22.017983 19026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:57:22.019403 19011 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 00:57:22.019564 19011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/wal
uuid: "13e4c7a19c9a4311bda3be760a64a57f"
format_stamp: "Formatted at 2026-09-24 00:57:22 on dist-test-slave-mmhs"
server_key: "f2c75f0f9c67bd8e56bcba369ce68346"
server_key_iv: "7325124fdf5b925e4530a76c11ca15f9"
server_key_version: "encryptionkey@0"
I20260924 00:57:22.019757 19011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:57:22.035182 19011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:57:22.039137 19011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:57:22.039430 19011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:57:22.040367 19011 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:57:22.041689 19011 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:57:22.041769 19011 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:57:22.041915 19011 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:57:22.041970 19011 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:57:22.075477 19011 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.130:37071
I20260924 00:57:22.075513 19138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.130:37071 every 8 connection(s)
I20260924 00:57:22.076793 19011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
I20260924 00:57:22.077893 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 19011
I20260924 00:57:22.078119 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance
I20260924 00:57:22.078649 8450 external_mini_cluster.cc:1428] Setting key d8ed7525b64d97a47c96901cb6cca96c
Sep 24 00:57:22 dist-test-slave-mmhs krb5kdc[18771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211442, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.130@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:57:22.099499 19007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:22.079780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:35233 (local address 127.8.64.190:33751)
0924 00:57:22.079988 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:22.079992 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:22.081461 (+ 1469us) server_negotiation.cc:407] Connection header received
0924 00:57:22.083628 (+ 2167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:22.083634 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:22.083756 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:22.083875 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:22.085448 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:22.086201 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:22.087872 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:22.088332 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:22.088480 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:22.092361 (+ 3881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:57:22.092370 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:57:22.092378 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:57:22.092423 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:57:22.096152 (+ 3729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:57:22.096903 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:57:22.096911 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:57:22.096917 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:57:22.097086 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:57:22.097520 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:57:22.097528 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:57:22.097534 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:57:22.097920 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:57:22.098076 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0924 00:57:22.098927 (+ 851us) server_negotiation.cc:299] Negotiation successful
0924 00:57:22.099155 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260924 00:57:22.100616 19141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:22.080450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:33751 (local address 127.8.64.130:35233)
0924 00:57:22.081313 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:22.081351 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:22.083335 (+ 1984us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:22.084164 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:22.084173 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:22.084701 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:22.085278 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:22.085296 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:22.086394 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:22.086400 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:22.087720 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:22.087732 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:22.088579 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:22.088587 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:22.088834 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:22.089556 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:57:22.089580 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:22.092120 (+ 2540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:57:22.096317 (+ 4197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:57:22.096324 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:57:22.096338 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:57:22.096760 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:57:22.097201 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:57:22.097205 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:57:22.097207 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:57:22.097388 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:57:22.098106 (+ 718us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:57:22.098112 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:57:22.098544 (+ 432us) client_negotiation.cc:769] Sending connection context
0924 00:57:22.098814 (+ 270us) client_negotiation.cc:240] Negotiation successful
0924 00:57:22.099352 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":361,"threads_started":1}
I20260924 00:57:22.102237 19139 heartbeater.cc:347] Connected to a master server at 127.8.64.190:33751
I20260924 00:57:22.102603 19139 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.8.64.131@KRBTEST.COM; defaulting to no policy
I20260924 00:57:22.103377 19139 heartbeater.cc:511] Master 127.8.64.190:33751 requested a full tablet report, sending...
I20260924 00:57:22.104835 18816 ts_manager.cc:194] Registered new tserver with Master: 13e4c7a19c9a4311bda3be760a64a57f (127.8.64.130:37071)
I20260924 00:57:22.105795 18816 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.130@KRBTEST.COM'} at 127.8.64.130:35233
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab.
Entry for principal kudu/127.8.64.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab.
I20260924 00:57:22.135959 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.131:0
--local_ip_for_outbound_sockets=127.8.64.131
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:33751
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab
--principal=kudu/127.8.64.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:43283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:57:22.457536 19144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:57:22.457831 19144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:57:22.458003 19144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:57:22.465211 19144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:57:22.465358 19144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:57:22.465543 19144 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.131
I20260924 00:57:22.475008 19144 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:43283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab
--principal=kudu/127.8.64.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:33751
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.131
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:57:22.476286 19144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:57:22.477627 19144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:57:22.490465 19149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:57:22.490427 19150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:57:22.490517 19152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:57:22.491657 19144 server_base.cc:1034] running on GCE node
I20260924 00:57:22.492347 19144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:57:22.493254 19144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:57:22.494474 19144 hybrid_clock.cc:648] HybridClock initialized: now 1790211442494428 us; error 58 us; skew 500 ppm
I20260924 00:57:22.494786 19144 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:57:22 dist-test-slave-mmhs krb5kdc[18771](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211442, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:57:22.498016 19144 init.cc:377] Logged in from keytab as kudu/127.8.64.131@KRBTEST.COM (short username kudu)
I20260924 00:57:22.499584 19144 webserver.cc:460] Webserver started at http://127.8.64.131:45171/ using document root <none> and password file <none>
I20260924 00:57:22.500283 19144 fs_manager.cc:359] Metadata directory not provided
I20260924 00:57:22.500456 19144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:57:22.500829 19144 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:57:22.504091 19144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/data/instance:
uuid: "17a425c899cb4c9cb2159839c2d3b66c"
format_stamp: "Formatted at 2026-09-24 00:57:22 on dist-test-slave-mmhs"
server_key: "ca8f7612fdcb462a68692bfad6b9cfe6"
server_key_iv: "a3377367c76cd6f4913cd80e32c09fae"
server_key_version: "encryptionkey@0"
I20260924 00:57:22.504873 19144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance:
uuid: "17a425c899cb4c9cb2159839c2d3b66c"
format_stamp: "Formatted at 2026-09-24 00:57:22 on dist-test-slave-mmhs"
server_key: "ca8f7612fdcb462a68692bfad6b9cfe6"
server_key_iv: "a3377367c76cd6f4913cd80e32c09fae"
server_key_version: "encryptionkey@0"
I20260924 00:57:22.510062 19144 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260924 00:57:22.513895 19159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:57:22.515314 19144 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 00:57:22.515486 19144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/wal
uuid: "17a425c899cb4c9cb2159839c2d3b66c"
format_stamp: "Formatted at 2026-09-24 00:57:22 on dist-test-slave-mmhs"
server_key: "ca8f7612fdcb462a68692bfad6b9cfe6"
server_key_iv: "a3377367c76cd6f4913cd80e32c09fae"
server_key_version: "encryptionkey@0"
I20260924 00:57:22.515676 19144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:57:22.529029 19144 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:57:22.533444 19144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:57:22.533751 19144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:57:22.534767 19144 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:57:22.536059 19144 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:57:22.536149 19144 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:57:22.536294 19144 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:57:22.536360 19144 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:57:22.570631 19144 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.131:42215
I20260924 00:57:22.570670 19272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.131:42215 every 8 connection(s)
I20260924 00:57:22.572155 19144 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
I20260924 00:57:22.578605 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 19144
I20260924 00:57:22.578831 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance
I20260924 00:57:22.579246 8450 external_mini_cluster.cc:1428] Setting key e0a55c38d7e16c00424301d0fc93e5cc
Sep 24 00:57:22 dist-test-slave-mmhs krb5kdc[18771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211442, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.131@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:57:22.592084 19007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:22.574974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:34603 (local address 127.8.64.190:33751)
0924 00:57:22.575180 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:22.575187 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:22.576375 (+ 1188us) server_negotiation.cc:407] Connection header received
0924 00:57:22.578132 (+ 1757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:22.578138 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:22.578225 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:22.578372 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:22.579990 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:22.581009 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:22.582722 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:22.583183 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:22.583283 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:22.586665 (+ 3382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:57:22.586670 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:57:22.586673 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:57:22.586703 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:57:22.589227 (+ 2524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:57:22.589845 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:57:22.589848 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:57:22.589850 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:57:22.589969 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:57:22.590401 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:57:22.590410 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:57:22.590412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:57:22.590690 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:57:22.590848 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0924 00:57:22.591535 (+ 687us) server_negotiation.cc:299] Negotiation successful
0924 00:57:22.591741 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260924 00:57:22.593255 19275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:22.575488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:33751 (local address 127.8.64.131:34603)
0924 00:57:22.576218 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:22.576263 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:22.577855 (+ 1592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:22.578619 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:22.578628 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:22.579209 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:57:22.579809 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:22.579826 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:22.581165 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:22.581172 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:22.582595 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:22.582607 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:22.583363 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:22.583368 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:22.583558 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:22.584261 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:57:22.584282 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:57:22.586450 (+ 2168us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:57:22.589349 (+ 2899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:57:22.589355 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:57:22.589367 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:57:22.589748 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:57:22.590059 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:57:22.590062 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:57:22.590064 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:57:22.590227 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:57:22.590820 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:57:22.590828 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:57:22.591161 (+ 333us) client_negotiation.cc:769] Sending connection context
0924 00:57:22.591410 (+ 249us) client_negotiation.cc:240] Negotiation successful
0924 00:57:22.591908 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":284,"threads_started":1}
I20260924 00:57:22.594656 19273 heartbeater.cc:347] Connected to a master server at 127.8.64.190:33751
I20260924 00:57:22.594921 19273 heartbeater.cc:464] Registering TS with master...
I20260924 00:57:22.595587 19273 heartbeater.cc:511] Master 127.8.64.190:33751 requested a full tablet report, sending...
I20260924 00:57:22.596912 18815 ts_manager.cc:194] Registered new tserver with Master: 17a425c899cb4c9cb2159839c2d3b66c (127.8.64.131:42215)
I20260924 00:57:22.597777 18815 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.131@KRBTEST.COM'} at 127.8.64.131:34603
I20260924 00:57:22.605175 19005 heartbeater.cc:503] Master 127.8.64.190:33751 was elected leader, sending a full tablet report...
I20260924 00:57:22.606544 8450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 00:57:22.622776 19007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:22.611052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50626 (local address 127.8.64.190:33751)
0924 00:57:22.611205 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:22.611210 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:22.611714 (+ 504us) server_negotiation.cc:407] Connection header received
0924 00:57:22.611834 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:22.611837 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:22.611893 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:22.611972 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:22.613269 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:22.614098 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:22.615348 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:22.615735 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:22.615835 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:22.617949 (+ 2114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:57:22.617953 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:57:22.617957 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:57:22.617984 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:57:22.620532 (+ 2548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:57:22.621105 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:57:22.621109 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:57:22.621111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:57:22.621227 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:57:22.621515 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:57:22.621518 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:57:22.621520 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:57:22.621790 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:57:22.621963 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0924 00:57:22.622236 (+ 273us) server_negotiation.cc:299] Negotiation successful
0924 00:57:22.622440 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260924 00:57:22.626921 18815 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50626:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:57:22.633610 18815 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:57:22.661242 19289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:22.646917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:45401 (local address 127.0.0.1:53558)
0924 00:57:22.650905 (+ 3988us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:22.650928 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:22.651089 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:22.652253 (+ 1164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:22.652265 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:22.652305 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:57:22.654013 (+ 1708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:22.654037 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:22.655590 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:22.655595 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:22.658070 (+ 2475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:22.658080 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:22.660288 (+ 2208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:22.660294 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:22.660563 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:22.660627 (+ 64us) client_negotiation.cc:769] Sending connection context
0924 00:57:22.660753 (+ 126us) client_negotiation.cc:240] Negotiation successful
0924 00:57:22.660911 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3718,"mutex_wait_us":50,"thread_start_us":355,"threads_started":1}
I20260924 00:57:22.662313 19287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:22.646971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53558 (local address 127.8.64.129:45401)
0924 00:57:22.649664 (+ 2693us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:22.649674 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:22.651238 (+ 1564us) server_negotiation.cc:407] Connection header received
0924 00:57:22.651366 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:22.651377 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:22.651701 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:22.651916 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:57:22.654261 (+ 2345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:22.655450 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:22.658219 (+ 2769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:22.659784 (+ 1565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:22.659918 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:22.660152 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0924 00:57:22.661502 (+ 1350us) server_negotiation.cc:299] Negotiation successful
0924 00:57:22.661973 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2521,"thread_start_us":2345,"threads_started":1}
I20260924 00:57:22.663564 19288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:22.647412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43230 (local address 127.8.64.130:37071)
0924 00:57:22.655280 (+ 7868us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:22.655292 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:22.655327 (+ 35us) server_negotiation.cc:407] Connection header received
0924 00:57:22.655450 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:22.655461 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:22.655752 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:22.655993 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:57:22.656894 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:22.658466 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:22.660799 (+ 2333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:22.662247 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:22.662471 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:22.662644 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0924 00:57:22.663134 (+ 490us) server_negotiation.cc:299] Negotiation successful
0924 00:57:22.663292 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7728,"thread_start_us":7635,"threads_started":1}
I20260924 00:57:22.663566 19286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:22.646843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.130:37071 (local address 127.0.0.1:43230)
0924 00:57:22.649725 (+ 2882us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:22.649769 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:22.650975 (+ 1206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:22.655975 (+ 5000us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:22.655979 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:22.656018 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:57:22.656675 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:22.656688 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:22.658591 (+ 1903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:22.658597 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:22.660631 (+ 2034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:22.660641 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:22.662764 (+ 2123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:22.662771 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:22.662932 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:22.662965 (+ 33us) client_negotiation.cc:769] Sending connection context
0924 00:57:22.663073 (+ 108us) client_negotiation.cc:240] Negotiation successful
0924 00:57:22.663192 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"mutex_wait_us":35,"thread_start_us":9620,"threads_started":1}
I20260924 00:57:22.665668 18940 tablet_service.cc:1467] Processing CreateTablet for tablet 69f47261efac4a50a626371e7c50695b (DEFAULT_TABLE table=test-table [id=d7f579d0840447b583d842f0bd2efe8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:57:22.666908 18940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69f47261efac4a50a626371e7c50695b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:57:22.668722 19285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:22.648233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.131:42215 (local address 127.0.0.1:36638)
0924 00:57:22.656836 (+ 8603us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:22.656854 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:22.656976 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:22.657837 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:22.657844 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:22.657868 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:57:22.658325 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:22.658335 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:22.659697 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:22.659704 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:22.661824 (+ 2120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:22.661839 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:22.667934 (+ 6095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:22.667941 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:22.668107 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:22.668137 (+ 30us) client_negotiation.cc:769] Sending connection context
0924 00:57:22.668223 (+ 86us) client_negotiation.cc:240] Negotiation successful
0924 00:57:22.668360 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8442,"mutex_wait_us":76,"thread_start_us":2237,"threads_started":1}
I20260924 00:57:22.669358 19290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:22.648809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36638 (local address 127.8.64.131:42215)
0924 00:57:22.654880 (+ 6071us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:22.654891 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:22.657112 (+ 2221us) server_negotiation.cc:407] Connection header received
0924 00:57:22.657211 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:22.657221 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:22.657431 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:22.657596 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:57:22.658513 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:22.659538 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:22.662010 (+ 2472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:22.667408 (+ 5398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:22.667569 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:22.667795 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0924 00:57:22.669004 (+ 1209us) server_negotiation.cc:299] Negotiation successful
0924 00:57:22.669129 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5860,"thread_start_us":5696,"threads_started":1}
I20260924 00:57:22.670854 19074 tablet_service.cc:1467] Processing CreateTablet for tablet 69f47261efac4a50a626371e7c50695b (DEFAULT_TABLE table=test-table [id=d7f579d0840447b583d842f0bd2efe8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:57:22.672237 19074 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69f47261efac4a50a626371e7c50695b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:57:22.672075 19207 tablet_service.cc:1467] Processing CreateTablet for tablet 69f47261efac4a50a626371e7c50695b (DEFAULT_TABLE table=test-table [id=d7f579d0840447b583d842f0bd2efe8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:57:22.673134 19207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69f47261efac4a50a626371e7c50695b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:57:22.683058 19291 tablet_bootstrap.cc:492] T 69f47261efac4a50a626371e7c50695b P 14258d929a9d4993846af747a2ba1d7c: Bootstrap starting.
I20260924 00:57:22.686674 19293 tablet_bootstrap.cc:492] T 69f47261efac4a50a626371e7c50695b P 13e4c7a19c9a4311bda3be760a64a57f: Bootstrap starting.
I20260924 00:57:22.687536 19291 tablet_bootstrap.cc:654] T 69f47261efac4a50a626371e7c50695b P 14258d929a9d4993846af747a2ba1d7c: Neither blocks nor log segments found. Creating new log.
I20260924 00:57:22.688032 19292 tablet_bootstrap.cc:492] T 69f47261efac4a50a626371e7c50695b P 17a425c899cb4c9cb2159839c2d3b66c: Bootstrap starting.
I20260924 00:57:22.688894 19291 log.cc:824] T 69f47261efac4a50a626371e7c50695b P 14258d929a9d4993846af747a2ba1d7c: Log is configured to *not* fsync() on all Append() calls
I20260924 00:57:22.690475 19293 tablet_bootstrap.cc:654] T 69f47261efac4a50a626371e7c50695b P 13e4c7a19c9a4311bda3be760a64a57f: Neither blocks nor log segments found. Creating new log.
I20260924 00:57:22.691565 19292 tablet_bootstrap.cc:654] T 69f47261efac4a50a626371e7c50695b P 17a425c899cb4c9cb2159839c2d3b66c: Neither blocks nor log segments found. Creating new log.
I20260924 00:57:22.691700 19291 tablet_bootstrap.cc:492] T 69f47261efac4a50a626371e7c50695b P 14258d929a9d4993846af747a2ba1d7c: No bootstrap required, opened a new log
I20260924 00:57:22.691768 19293 log.cc:824] T 69f47261efac4a50a626371e7c50695b P 13e4c7a19c9a4311bda3be760a64a57f: Log is configured to *not* fsync() on all Append() calls
I20260924 00:57:22.691988 19291 ts_tablet_manager.cc:1397] T 69f47261efac4a50a626371e7c50695b P 14258d929a9d4993846af747a2ba1d7c: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260924 00:57:22.692799 19292 log.cc:824] T 69f47261efac4a50a626371e7c50695b P 17a425c899cb4c9cb2159839c2d3b66c: Log is configured to *not* fsync() on all Append() calls
I20260924 00:57:22.694912 19293 tablet_bootstrap.cc:492] T 69f47261efac4a50a626371e7c50695b P 13e4c7a19c9a4311bda3be760a64a57f: No bootstrap required, opened a new log
I20260924 00:57:22.695240 19293 ts_tablet_manager.cc:1397] T 69f47261efac4a50a626371e7c50695b P 13e4c7a19c9a4311bda3be760a64a57f: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260924 00:57:22.696343 19292 tablet_bootstrap.cc:492] T 69f47261efac4a50a626371e7c50695b P 17a425c899cb4c9cb2159839c2d3b66c: No bootstrap required, opened a new log
I20260924 00:57:22.696604 19292 ts_tablet_manager.cc:1397] T 69f47261efac4a50a626371e7c50695b P 17a425c899cb4c9cb2159839c2d3b66c: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260924 00:57:22.702857 19293 raft_consensus.cc:348] T 69f47261efac4a50a626371e7c50695b P 13e4c7a19c9a4311bda3be760a64a57f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14258d929a9d4993846af747a2ba1d7c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 45401 } } peers { permanent_uuid: "13e4c7a19c9a4311bda3be760a64a57f" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 37071 } } peers { permanent_uuid: "17a425c899cb4c9cb2159839c2d3b66c" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 42215 } }
I20260924 00:57:22.702863 19291 raft_consensus.cc:348] T 69f47261efac4a50a626371e7c50695b P 14258d929a9d4993846af747a2ba1d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14258d929a9d4993846af747a2ba1d7c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 45401 } } peers { permanent_uuid: "13e4c7a19c9a4311bda3be760a64a57f" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 37071 } } peers { permanent_uuid: "17a425c899cb4c9cb2159839c2d3b66c" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 42215 } }
I20260924 00:57:22.703263 19293 raft_consensus.cc:374] T 69f47261efac4a50a626371e7c50695b P 13e4c7a19c9a4311bda3be760a64a57f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:57:22.703286 19291 raft_consensus.cc:374] T 69f47261efac4a50a626371e7c50695b P 14258d929a9d4993846af747a2ba1d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:57:22.703357 19293 raft_consensus.cc:729] T 69f47261efac4a50a626371e7c50695b P 13e4c7a19c9a4311bda3be760a64a57f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13e4c7a19c9a4311bda3be760a64a57f, State: Initialized, Role: FOLLOWER
I20260924 00:57:22.703372 19291 raft_consensus.cc:729] T 69f47261efac4a50a626371e7c50695b P 14258d929a9d4993846af747a2ba1d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14258d929a9d4993846af747a2ba1d7c, State: Initialized, Role: FOLLOWER
I20260924 00:57:22.704026 19293 consensus_queue.cc:260] T 69f47261efac4a50a626371e7c50695b P 13e4c7a19c9a4311bda3be760a64a57f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14258d929a9d4993846af747a2ba1d7c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 45401 } } peers { permanent_uuid: "13e4c7a19c9a4311bda3be760a64a57f" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 37071 } } peers { permanent_uuid: "17a425c899cb4c9cb2159839c2d3b66c" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 42215 } }
I20260924 00:57:22.703992 19291 consensus_queue.cc:260] T 69f47261efac4a50a626371e7c50695b P 14258d929a9d4993846af747a2ba1d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14258d929a9d4993846af747a2ba1d7c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 45401 } } peers { permanent_uuid: "13e4c7a19c9a4311bda3be760a64a57f" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 37071 } } peers { permanent_uuid: "17a425c899cb4c9cb2159839c2d3b66c" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 42215 } }
I20260924 00:57:22.705662 19139 heartbeater.cc:503] Master 127.8.64.190:33751 was elected leader, sending a full tablet report...
I20260924 00:57:22.705849 19291 ts_tablet_manager.cc:1428] T 69f47261efac4a50a626371e7c50695b P 14258d929a9d4993846af747a2ba1d7c: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260924 00:57:22.707329 19293 ts_tablet_manager.cc:1428] T 69f47261efac4a50a626371e7c50695b P 13e4c7a19c9a4311bda3be760a64a57f: Time spent starting tablet: real 0.012s user 0.007s sys 0.003s
I20260924 00:57:22.707588 19292 raft_consensus.cc:348] T 69f47261efac4a50a626371e7c50695b P 17a425c899cb4c9cb2159839c2d3b66c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14258d929a9d4993846af747a2ba1d7c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 45401 } } peers { permanent_uuid: "13e4c7a19c9a4311bda3be760a64a57f" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 37071 } } peers { permanent_uuid: "17a425c899cb4c9cb2159839c2d3b66c" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 42215 } }
I20260924 00:57:22.708133 19292 raft_consensus.cc:374] T 69f47261efac4a50a626371e7c50695b P 17a425c899cb4c9cb2159839c2d3b66c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:57:22.708238 19292 raft_consensus.cc:729] T 69f47261efac4a50a626371e7c50695b P 17a425c899cb4c9cb2159839c2d3b66c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17a425c899cb4c9cb2159839c2d3b66c, State: Initialized, Role: FOLLOWER
I20260924 00:57:22.709053 19292 consensus_queue.cc:260] T 69f47261efac4a50a626371e7c50695b P 17a425c899cb4c9cb2159839c2d3b66c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14258d929a9d4993846af747a2ba1d7c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 45401 } } peers { permanent_uuid: "13e4c7a19c9a4311bda3be760a64a57f" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 37071 } } peers { permanent_uuid: "17a425c899cb4c9cb2159839c2d3b66c" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 42215 } }
I20260924 00:57:22.715066 19273 heartbeater.cc:503] Master 127.8.64.190:33751 was elected leader, sending a full tablet report...
I20260924 00:57:22.715583 19292 ts_tablet_manager.cc:1428] T 69f47261efac4a50a626371e7c50695b P 17a425c899cb4c9cb2159839c2d3b66c: Time spent starting tablet: real 0.019s user 0.009s sys 0.010s
I20260924 00:57:22.752314 19299 raft_consensus.cc:482] T 69f47261efac4a50a626371e7c50695b P 17a425c899cb4c9cb2159839c2d3b66c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:57:22.752645 19299 raft_consensus.cc:504] T 69f47261efac4a50a626371e7c50695b P 17a425c899cb4c9cb2159839c2d3b66c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14258d929a9d4993846af747a2ba1d7c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 45401 } } peers { permanent_uuid: "13e4c7a19c9a4311bda3be760a64a57f" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 37071 } } peers { permanent_uuid: "17a425c899cb4c9cb2159839c2d3b66c" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 42215 } }
I20260924 00:57:22.754428 19299 leader_election.cc:288] T 69f47261efac4a50a626371e7c50695b P 17a425c899cb4c9cb2159839c2d3b66c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 14258d929a9d4993846af747a2ba1d7c (127.8.64.129:45401), 13e4c7a19c9a4311bda3be760a64a57f (127.8.64.130:37071)
I20260924 00:57:22.761603 19287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:22.754839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:37659 (local address 127.8.64.129:45401)
0924 00:57:22.755040 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:22.755050 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:22.755095 (+ 45us) server_negotiation.cc:407] Connection header received
0924 00:57:22.755182 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:22.755190 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:22.755291 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:22.755404 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:57:22.755908 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:22.756983 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:22.759724 (+ 2741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:22.760634 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:22.760760 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:22.760813 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 00:57:22.761231 (+ 418us) server_negotiation.cc:299] Negotiation successful
0924 00:57:22.761360 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260924 00:57:22.761596 19275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:22.754597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:45401 (local address 127.8.64.131:37659)
0924 00:57:22.754793 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:22.754815 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:22.754941 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:22.755449 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:22.755457 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:22.755510 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:57:22.755756 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:22.755769 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:22.757140 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:22.757145 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:22.759551 (+ 2406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:22.759566 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:22.760801 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:22.760808 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:22.761028 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:22.761069 (+ 41us) client_negotiation.cc:769] Sending connection context
0924 00:57:22.761184 (+ 115us) client_negotiation.cc:240] Negotiation successful
0924 00:57:22.761335 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":49}
I20260924 00:57:22.762125 19288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:22.755089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:37651 (local address 127.8.64.130:37071)
0924 00:57:22.755296 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:22.755303 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:22.755768 (+ 465us) server_negotiation.cc:407] Connection header received
0924 00:57:22.755917 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:22.755922 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:22.756001 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:22.756113 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:57:22.756528 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:22.757415 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:22.760091 (+ 2676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:22.761092 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:22.761241 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:22.761312 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0924 00:57:22.761714 (+ 402us) server_negotiation.cc:299] Negotiation successful
0924 00:57:22.761852 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260924 00:57:22.762192 19300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:22.754885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.130:37071 (local address 127.8.64.131:37651)
0924 00:57:22.755642 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0924 00:57:22.755664 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0924 00:57:22.755784 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:57:22.756122 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:57:22.756124 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:57:22.756143 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:57:22.756387 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:22.756394 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:22.757580 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:22.757587 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:22.759924 (+ 2337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:57:22.759940 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:57:22.761304 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:57:22.761311 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:57:22.761492 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:22.761523 (+ 31us) client_negotiation.cc:769] Sending connection context
0924 00:57:22.761652 (+ 129us) client_negotiation.cc:240] Negotiation successful
0924 00:57:22.761781 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"mutex_wait_us":24,"thread_start_us":424,"threads_started":1}
I20260924 00:57:22.763166 19094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69f47261efac4a50a626371e7c50695b" candidate_uuid: "17a425c899cb4c9cb2159839c2d3b66c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13e4c7a19c9a4311bda3be760a64a57f" is_pre_election: true
I20260924 00:57:22.763209 18960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69f47261efac4a50a626371e7c50695b" candidate_uuid: "17a425c899cb4c9cb2159839c2d3b66c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14258d929a9d4993846af747a2ba1d7c" is_pre_election: true
I20260924 00:57:22.763676 19094 raft_consensus.cc:2454] T 69f47261efac4a50a626371e7c50695b P 13e4c7a19c9a4311bda3be760a64a57f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17a425c899cb4c9cb2159839c2d3b66c in term 0.
I20260924 00:57:22.763768 18960 raft_consensus.cc:2454] T 69f47261efac4a50a626371e7c50695b P 14258d929a9d4993846af747a2ba1d7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17a425c899cb4c9cb2159839c2d3b66c in term 0.
I20260924 00:57:22.764374 19160 leader_election.cc:302] T 69f47261efac4a50a626371e7c50695b P 17a425c899cb4c9cb2159839c2d3b66c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13e4c7a19c9a4311bda3be760a64a57f, 17a425c899cb4c9cb2159839c2d3b66c; no voters:
I20260924 00:57:22.764791 19299 raft_consensus.cc:2781] T 69f47261efac4a50a626371e7c50695b P 17a425c899cb4c9cb2159839c2d3b66c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:57:22.764940 19299 raft_consensus.cc:482] T 69f47261efac4a50a626371e7c50695b P 17a425c899cb4c9cb2159839c2d3b66c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:57:22.765072 19299 raft_consensus.cc:3037] T 69f47261efac4a50a626371e7c50695b P 17a425c899cb4c9cb2159839c2d3b66c [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:57:22.767571 19299 raft_consensus.cc:504] T 69f47261efac4a50a626371e7c50695b P 17a425c899cb4c9cb2159839c2d3b66c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14258d929a9d4993846af747a2ba1d7c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 45401 } } peers { permanent_uuid: "13e4c7a19c9a4311bda3be760a64a57f" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 37071 } } peers { permanent_uuid: "17a425c899cb4c9cb2159839c2d3b66c" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 42215 } }
I20260924 00:57:22.768275 19299 leader_election.cc:288] T 69f47261efac4a50a626371e7c50695b P 17a425c899cb4c9cb2159839c2d3b66c [CANDIDATE]: Term 1 election: Requested vote from peers 14258d929a9d4993846af747a2ba1d7c (127.8.64.129:45401), 13e4c7a19c9a4311bda3be760a64a57f (127.8.64.130:37071)
I20260924 00:57:22.768684 18960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69f47261efac4a50a626371e7c50695b" candidate_uuid: "17a425c899cb4c9cb2159839c2d3b66c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14258d929a9d4993846af747a2ba1d7c"
I20260924 00:57:22.768785 19094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69f47261efac4a50a626371e7c50695b" candidate_uuid: "17a425c899cb4c9cb2159839c2d3b66c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13e4c7a19c9a4311bda3be760a64a57f"
I20260924 00:57:22.768906 18960 raft_consensus.cc:3037] T 69f47261efac4a50a626371e7c50695b P 14258d929a9d4993846af747a2ba1d7c [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:57:22.769011 19094 raft_consensus.cc:3037] T 69f47261efac4a50a626371e7c50695b P 13e4c7a19c9a4311bda3be760a64a57f [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:57:22.772434 18960 raft_consensus.cc:2454] T 69f47261efac4a50a626371e7c50695b P 14258d929a9d4993846af747a2ba1d7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17a425c899cb4c9cb2159839c2d3b66c in term 1.
I20260924 00:57:22.772434 19094 raft_consensus.cc:2454] T 69f47261efac4a50a626371e7c50695b P 13e4c7a19c9a4311bda3be760a64a57f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17a425c899cb4c9cb2159839c2d3b66c in term 1.
I20260924 00:57:22.772852 19161 leader_election.cc:302] T 69f47261efac4a50a626371e7c50695b P 17a425c899cb4c9cb2159839c2d3b66c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14258d929a9d4993846af747a2ba1d7c, 17a425c899cb4c9cb2159839c2d3b66c; no voters:
I20260924 00:57:22.773161 19299 raft_consensus.cc:2781] T 69f47261efac4a50a626371e7c50695b P 17a425c899cb4c9cb2159839c2d3b66c [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:57:22.773615 19299 raft_consensus.cc:686] T 69f47261efac4a50a626371e7c50695b P 17a425c899cb4c9cb2159839c2d3b66c [term 1 LEADER]: Becoming Leader. State: Replica: 17a425c899cb4c9cb2159839c2d3b66c, State: Running, Role: LEADER
I20260924 00:57:22.774089 19299 consensus_queue.cc:237] T 69f47261efac4a50a626371e7c50695b P 17a425c899cb4c9cb2159839c2d3b66c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14258d929a9d4993846af747a2ba1d7c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 45401 } } peers { permanent_uuid: "13e4c7a19c9a4311bda3be760a64a57f" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 37071 } } peers { permanent_uuid: "17a425c899cb4c9cb2159839c2d3b66c" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 42215 } }
I20260924 00:57:22.779624 18816 catalog_manager.cc:5495] T 69f47261efac4a50a626371e7c50695b P 17a425c899cb4c9cb2159839c2d3b66c reported cstate change: term changed from 0 to 1, leader changed from <none> to 17a425c899cb4c9cb2159839c2d3b66c (127.8.64.131). New cstate: current_term: 1 leader_uuid: "17a425c899cb4c9cb2159839c2d3b66c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14258d929a9d4993846af747a2ba1d7c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 45401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13e4c7a19c9a4311bda3be760a64a57f" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 37071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17a425c899cb4c9cb2159839c2d3b66c" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 42215 } health_report { overall_health: HEALTHY } } }
I20260924 00:57:22.809584 19290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:22.804358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36654 (local address 127.8.64.131:42215)
0924 00:57:22.804558 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:22.804564 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:22.804585 (+ 21us) server_negotiation.cc:407] Connection header received
0924 00:57:22.804815 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:22.804819 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:22.804899 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:22.805049 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:57:22.805406 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:22.806145 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:22.807452 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:22.807891 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:22.808019 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:22.808485 (+ 466us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:57:22.809027 (+ 542us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:57:22.809196 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0924 00:57:22.809251 (+ 55us) server_negotiation.cc:299] Negotiation successful
0924 00:57:22.809327 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260924 00:57:22.817498 19094 raft_consensus.cc:1261] T 69f47261efac4a50a626371e7c50695b P 13e4c7a19c9a4311bda3be760a64a57f [term 1 FOLLOWER]: Refusing update from remote peer 17a425c899cb4c9cb2159839c2d3b66c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:57:22.817494 18960 raft_consensus.cc:1261] T 69f47261efac4a50a626371e7c50695b P 14258d929a9d4993846af747a2ba1d7c [term 1 FOLLOWER]: Refusing update from remote peer 17a425c899cb4c9cb2159839c2d3b66c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:57:22.818519 19301 consensus_queue.cc:1035] T 69f47261efac4a50a626371e7c50695b P 17a425c899cb4c9cb2159839c2d3b66c [LEADER]: Connected to new peer: Peer: permanent_uuid: "14258d929a9d4993846af747a2ba1d7c" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 45401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:57:22.818876 19299 consensus_queue.cc:1035] T 69f47261efac4a50a626371e7c50695b P 17a425c899cb4c9cb2159839c2d3b66c [LEADER]: Connected to new peer: Peer: permanent_uuid: "13e4c7a19c9a4311bda3be760a64a57f" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 37071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260924 00:57:22.822770 19006 tablet.cc:2369] T 69f47261efac4a50a626371e7c50695b P 14258d929a9d4993846af747a2ba1d7c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 00:57:22.823864 19274 tablet.cc:2369] T 69f47261efac4a50a626371e7c50695b P 17a425c899cb4c9cb2159839c2d3b66c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 00:57:22.829035 19140 tablet.cc:2369] T 69f47261efac4a50a626371e7c50695b P 13e4c7a19c9a4311bda3be760a64a57f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:57:22.833485 19304 mvcc.cc:205] Tried to move back new op lower bound from 7332706069771104256 to 7332706069606449152. Current Snapshot: MvccSnapshot[applied={T|T < 7332706069771104256}]
I20260924 00:57:22.841898 19305 mvcc.cc:205] Tried to move back new op lower bound from 7332706069771104256 to 7332706069606449152. Current Snapshot: MvccSnapshot[applied={T|T < 7332706069771104256}]
I20260924 00:57:22.849047 18816 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:57:22.849320 18816 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:57:22.852909 18816 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a1b932c316da4d549f0c193e9155fe2e: Sending DeleteTablet for 3 replicas of tablet 69f47261efac4a50a626371e7c50695b
I20260924 00:57:22.854138 18940 tablet_service.cc:1514] Processing DeleteTablet for tablet 69f47261efac4a50a626371e7c50695b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:57:22 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:53558
I20260924 00:57:22.854763 19207 tablet_service.cc:1514] Processing DeleteTablet for tablet 69f47261efac4a50a626371e7c50695b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:57:22 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:36638
I20260924 00:57:22.854264 19074 tablet_service.cc:1514] Processing DeleteTablet for tablet 69f47261efac4a50a626371e7c50695b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:57:22 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:43230
I20260924 00:57:22.856055 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 18877
I20260924 00:57:22.856117 19316 tablet_replica.cc:330] T 69f47261efac4a50a626371e7c50695b P 17a425c899cb4c9cb2159839c2d3b66c: stopping tablet replica
I20260924 00:57:22.856576 19316 raft_consensus.cc:2229] T 69f47261efac4a50a626371e7c50695b P 17a425c899cb4c9cb2159839c2d3b66c [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:57:22.857257 19316 raft_consensus.cc:2258] T 69f47261efac4a50a626371e7c50695b P 17a425c899cb4c9cb2159839c2d3b66c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:57:22.858001 19315 tablet_replica.cc:330] T 69f47261efac4a50a626371e7c50695b P 14258d929a9d4993846af747a2ba1d7c: stopping tablet replica
I20260924 00:57:22.858434 19315 raft_consensus.cc:2229] T 69f47261efac4a50a626371e7c50695b P 14258d929a9d4993846af747a2ba1d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:57:22.858942 19315 raft_consensus.cc:2258] T 69f47261efac4a50a626371e7c50695b P 14258d929a9d4993846af747a2ba1d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:57:22.860088 19316 ts_tablet_manager.cc:1905] T 69f47261efac4a50a626371e7c50695b P 17a425c899cb4c9cb2159839c2d3b66c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:57:22.866456 19317 tablet_replica.cc:330] T 69f47261efac4a50a626371e7c50695b P 13e4c7a19c9a4311bda3be760a64a57f: stopping tablet replica
I20260924 00:57:22.866895 19317 raft_consensus.cc:2229] T 69f47261efac4a50a626371e7c50695b P 13e4c7a19c9a4311bda3be760a64a57f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:57:22.867412 19317 raft_consensus.cc:2258] T 69f47261efac4a50a626371e7c50695b P 13e4c7a19c9a4311bda3be760a64a57f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:57:22.868645 19315 ts_tablet_manager.cc:1905] T 69f47261efac4a50a626371e7c50695b P 14258d929a9d4993846af747a2ba1d7c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:57:22.869199 19317 ts_tablet_manager.cc:1905] T 69f47261efac4a50a626371e7c50695b P 13e4c7a19c9a4311bda3be760a64a57f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:57:22.869678 19316 ts_tablet_manager.cc:1918] T 69f47261efac4a50a626371e7c50695b P 17a425c899cb4c9cb2159839c2d3b66c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:57:22.869822 19316 log.cc:1196] T 69f47261efac4a50a626371e7c50695b P 17a425c899cb4c9cb2159839c2d3b66c: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-2/wal/wals/69f47261efac4a50a626371e7c50695b
I20260924 00:57:22.870481 19316 ts_tablet_manager.cc:1939] T 69f47261efac4a50a626371e7c50695b P 17a425c899cb4c9cb2159839c2d3b66c: Deleting consensus metadata
I20260924 00:57:22.872171 18801 catalog_manager.cc:4841] TS 17a425c899cb4c9cb2159839c2d3b66c (127.8.64.131:42215): tablet 69f47261efac4a50a626371e7c50695b (table test-table [id=d7f579d0840447b583d842f0bd2efe8e]) successfully deleted
I20260924 00:57:22.875551 19315 ts_tablet_manager.cc:1918] T 69f47261efac4a50a626371e7c50695b P 14258d929a9d4993846af747a2ba1d7c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:57:22.875706 19315 log.cc:1196] T 69f47261efac4a50a626371e7c50695b P 14258d929a9d4993846af747a2ba1d7c: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-0/wal/wals/69f47261efac4a50a626371e7c50695b
Sep 24 00:57:22 dist-test-slave-mmhs krb5kdc[18771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211440, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.129@KRBTEST.COM
I20260924 00:57:22.876247 19315 ts_tablet_manager.cc:1939] T 69f47261efac4a50a626371e7c50695b P 14258d929a9d4993846af747a2ba1d7c: Deleting consensus metadata
I20260924 00:57:22.877280 19317 ts_tablet_manager.cc:1918] T 69f47261efac4a50a626371e7c50695b P 13e4c7a19c9a4311bda3be760a64a57f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:57:22.877434 19317 log.cc:1196] T 69f47261efac4a50a626371e7c50695b P 13e4c7a19c9a4311bda3be760a64a57f: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790211343099518-8450-0/minicluster-data/ts-1/wal/wals/69f47261efac4a50a626371e7c50695b
I20260924 00:57:22.877957 19317 ts_tablet_manager.cc:1939] T 69f47261efac4a50a626371e7c50695b P 13e4c7a19c9a4311bda3be760a64a57f: Deleting consensus metadata
I20260924 00:57:22.878019 18801 catalog_manager.cc:4841] TS 14258d929a9d4993846af747a2ba1d7c (127.8.64.129:45401): tablet 69f47261efac4a50a626371e7c50695b (table test-table [id=d7f579d0840447b583d842f0bd2efe8e]) successfully deleted
I20260924 00:57:22.879771 18800 catalog_manager.cc:4841] TS 13e4c7a19c9a4311bda3be760a64a57f (127.8.64.130:37071): tablet 69f47261efac4a50a626371e7c50695b (table test-table [id=d7f579d0840447b583d842f0bd2efe8e]) successfully deleted
I20260924 00:57:22.884830 19287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:22.857512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53560 (local address 127.8.64.129:45401)
0924 00:57:22.857730 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:22.857739 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:22.859707 (+ 1968us) server_negotiation.cc:407] Connection header received
0924 00:57:22.859918 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:22.859923 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:22.860000 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:22.860094 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:22.861754 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:22.862908 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:22.864821 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:22.865457 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:22.865572 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:22.878012 (+ 12440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:57:22.878026 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:57:22.878043 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:57:22.878088 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:57:22.881702 (+ 3614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:57:22.882462 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:57:22.882471 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:57:22.882473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:57:22.882606 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:57:22.883235 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:57:22.883240 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:57:22.883246 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:57:22.883653 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:57:22.883965 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0924 00:57:22.884274 (+ 309us) server_negotiation.cc:299] Negotiation successful
0924 00:57:22.884484 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260924 00:57:22.885551 19000 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:57:22.976233 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 18877
I20260924 00:57:23.002966 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 19011
Sep 24 00:57:23 dist-test-slave-mmhs krb5kdc[18771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211440, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.130@KRBTEST.COM
I20260924 00:57:23.018276 19288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:23.003936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43234 (local address 127.8.64.130:37071)
0924 00:57:23.004134 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:23.004141 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:23.004165 (+ 24us) server_negotiation.cc:407] Connection header received
0924 00:57:23.004236 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:23.004240 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:23.004303 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:23.004456 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:23.005629 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:23.006447 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:23.007797 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:23.008175 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:23.008311 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:23.012481 (+ 4170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:57:23.012496 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:57:23.012513 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:57:23.012551 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:57:23.015566 (+ 3015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:57:23.016237 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:57:23.016244 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:57:23.016247 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:57:23.016365 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:57:23.016730 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:57:23.016738 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:57:23.016741 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:57:23.017114 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:57:23.017403 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0924 00:57:23.017722 (+ 319us) server_negotiation.cc:299] Negotiation successful
0924 00:57:23.017945 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260924 00:57:23.019018 19134 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:57:23.111122 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 19011
I20260924 00:57:23.129485 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 19144
Sep 24 00:57:23 dist-test-slave-mmhs krb5kdc[18771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211440, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.131@KRBTEST.COM
I20260924 00:57:23.146672 19290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:57:23.130371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36670 (local address 127.8.64.131:42215)
0924 00:57:23.130558 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0924 00:57:23.130564 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:57:23.130587 (+ 23us) server_negotiation.cc:407] Connection header received
0924 00:57:23.130672 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:57:23.130678 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:57:23.130757 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:57:23.130877 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:57:23.132065 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:23.133282 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:23.134763 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:57:23.135439 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:57:23.135577 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:57:23.139711 (+ 4134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:57:23.139731 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:57:23.139750 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:57:23.139793 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:57:23.143627 (+ 3834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:57:23.144281 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:57:23.144297 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:57:23.144302 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:57:23.144485 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:57:23.144864 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:57:23.144881 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:57:23.144885 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:57:23.145491 (+ 606us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:57:23.145821 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0924 00:57:23.146016 (+ 195us) server_negotiation.cc:299] Negotiation successful
0924 00:57:23.146250 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260924 00:57:23.147517 19268 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:57:23.268939 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 19144
I20260924 00:57:23.288017 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 18784
I20260924 00:57:23.289354 18846 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:57:23.396384 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 18784
2026-09-24T00:57:23Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4235 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:57:23 dist-test-slave-mmhs krb5kdc[19325](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:57:23 dist-test-slave-mmhs krb5kdc[19325](info): set up 2 sockets
Sep 24 00:57:23 dist-test-slave-mmhs krb5kdc[19325](info): commencing operation
krb5kdc: starting...
W20260924 00:57:24.504974 8450 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Sep 24 00:57:24 dist-test-slave-mmhs krb5kdc[19325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:57:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:57:24Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.8.64.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.8.64.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.8.64.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/krb5kdc/rangeradmin_127.8.64.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.8.64.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/krb5kdc/rangeradmin_127.8.64.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.8.64.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.8.64.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.8.64.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/krb5kdc/rangerlookup_127.8.64.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.8.64.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/krb5kdc/rangerlookup_127.8.64.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.64.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.8.64.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.64.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/krb5kdc/HTTP_127.8.64.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.64.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/krb5kdc/HTTP_127.8.64.148@KRBTEST.COM.keytab.
I20260924 00:57:24.853609 8450 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskDpZYBj/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/postgres -l logfile start
2026-09-24 00:57:27.665 UTC [19355] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-24 00:57:27.665 UTC [19355] LOG: listening on IPv4 address "127.8.64.148", port 44829
2026-09-24 00:57:27.668 UTC [19355] LOG: listening on Unix socket "/tmp/.s.PGSQL.44829"
2026-09-24 00:57:27.690 UTC [19358] LOG: database system was shut down at 2026-09-24 00:57:25 UTC
2026-09-24 00:57:27.694 UTC [19355] LOG: database system is ready to accept connections
I20260924 00:57:28.677248 8450 mini_postgres.cc:96] Postgres bound to 44829
2026-09-24 00:57:28.697 UTC [19366] FATAL: database "slave" does not exist
127.8.64.148:44829 - accepting connections
I20260924 00:57:28.698459 8450 mini_ranger.cc:161] Starting Ranger...
I20260924 00:57:28.725517 8450 mini_ranger.cc:84] Created miniranger Postgres user
I20260924 00:57:29.280179 8450 mini_ranger.cc:87] Created ranger Postgres database
I20260924 00:57:29.280316 8450 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ranger-admin
2026-09-24 00:57:29,577 [I] DB FLAVOR :POSTGRES
2026-09-24 00:57:29,578 [I] --------- Verifying Ranger DB connection ---------
2026-09-24 00:57:29,578 [I] Checking connection..
2026-09-24 00:57:29,578 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.64.148:44829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-24 00:57:29,850 [I] Checking connection passed.
2026-09-24 00:57:29,850 [I] --------- Verifying version history table ---------
2026-09-24 00:57:29,850 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.64.148:44829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 00:57:30,108 [I] Table x_db_version_h does not exist in database ranger
2026-09-24 00:57:30,109 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.64.148:44829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 00:57:30,369 [I] Table x_db_version_h does not exist in database ranger
2026-09-24 00:57:30,370 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-24 00:57:30,370 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.64.148:44829/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-24 00:57:30.609 UTC [19430] WARNING: there is no transaction in progress
2026-09-24 00:57:30,615 [I] create_dbversion_catalog.sql file imported successfully
2026-09-24 00:57:30,616 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.64.148:44829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 00:57:30,887 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-24 00:57:30,888 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-24 00:57:30,888 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.64.148:44829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-24 00:57:31,151 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.64.148:44829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-24 00:57:31,408 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.64.148:44829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-mmhs.c.gcp-upstream.internal','N') ;"
2026-09-24 00:57:31,658 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.64.148:44829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-24 00:57:31,925 [I] Table x_portal_user does not exist in database ranger
2026-09-24 00:57:31,925 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.64.148:44829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-24 00:57:32,165 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-24 00:57:32,165 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.64.148:44829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-24 00:57:32,394 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.64.148:44829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-24 00:57:32,640 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-24 00:57:32,641 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.64.148:44829/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-24 00:57:33.673 UTC [19539] WARNING: there is no transaction in progress
2026-09-24 00:57:33.691 UTC [19539] WARNING: there is no transaction in progress
2026-09-24 00:57:33.710 UTC [19539] WARNING: there is no transaction in progress
2026-09-24 00:57:33.727 UTC [19539] WARNING: there is no transaction in progress
2026-09-24 00:57:33.744 UTC [19539] WARNING: there is no transaction in progress
2026-09-24 00:57:33.764 UTC [19539] WARNING: there is no transaction in progress
2026-09-24 00:57:33.830 UTC [19539] WARNING: there is no transaction in progress
2026-09-24 00:57:33.837 UTC [19539] WARNING: there is no transaction in progress
2026-09-24 00:57:33.852 UTC [19539] WARNING: there is no transaction in progress
2026-09-24 00:57:33.864 UTC [19539] WARNING: there is no transaction in progress
2026-09-24 00:57:33.876 UTC [19539] WARNING: there is no transaction in progress
2026-09-24 00:57:33.892 UTC [19539] WARNING: there is no transaction in progress
2026-09-24 00:57:33.904 UTC [19539] WARNING: there is no transaction in progress
2026-09-24 00:57:33.911 UTC [19539] WARNING: there is no transaction in progress
2026-09-24 00:57:33.919 UTC [19539] WARNING: there is no transaction in progress
2026-09-24 00:57:33.927 UTC [19539] WARNING: there is no transaction in progress
2026-09-24 00:57:34,656 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-24 00:57:34,656 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.64.148:44829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-24 00:57:34,897 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-24 00:57:34,898 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.64.148:44829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-24 00:57:35,161 [I] Table x_portal_user already exists in database 'ranger'
2026-09-24 00:57:35,161 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.64.148:44829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-24 00:57:35,412 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-24 00:57:35,413 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.64.148:44829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-24 00:57:35,648 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.64.148:44829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-24 00:57:35,893 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.64.148:44829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-mmhs.c.gcp-upstream.internal';"
2026-09-24 00:57:36,126 [I] CORE_DB_SCHEMA import status has been updated
2026-09-24 00:57:36,126 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.64.148:44829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-24 00:57:36,358 [I] DB_PATCHES have already been applied
I20260924 00:57:36.364358 8450 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ranger-admin:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskDpZYBj/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260924 00:57:36.364498 8450 mini_ranger.cc:193] Using host: 127.8.64.148
I20260924 00:57:36.381364 8450 mini_ranger.cc:227] Ranger admin URL: http://127.8.64.148:53475
Sep 24, 2026 12:57:36 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 24, 2026 12:57:36 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 24, 2026 12:57:36 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ranger-admin/ews/webapp
Sep 24, 2026 12:57:36 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 24, 2026 12:57:36 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 24, 2026 12:57:37 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ranger-admin/ews/webapp].
Sep 24, 2026 12:57:37 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260924 00:57:37.539659 8450 mini_ranger.cc:160] Time spent starting Ranger: real 8.841s user 0.003s sys 0.043s
Sep 24, 2026 12:57:56 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 24, 2026 12:57:56 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Sep 24, 2026 12:57:56 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 24, 2026 12:57:56 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 24, 2026 12:57:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 24, 2026 12:57:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 24, 2026 12:57:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 24, 2026 12:57:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 24, 2026 12:57:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 24, 2026 12:57:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 24, 2026 12:57:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 24, 2026 12:57:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 24, 2026 12:57:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 24, 2026 12:57:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 24, 2026 12:57:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 24, 2026 12:57:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 24, 2026 12:57:56 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 24, 2026 12:57:57 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260924 00:57:58.274900 8450 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.8.64.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.8.64.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.8.64.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/krb5kdc/rangerkms_127.8.64.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.8.64.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/krb5kdc/rangerkms_127.8.64.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.64.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.8.64.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.64.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/krb5kdc/HTTP_127.8.64.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.64.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/krb5kdc/HTTP_127.8.64.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 24 00:57:58 dist-test-slave-mmhs krb5kdc[19325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211478, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260924 00:57:58.472157 8450 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260924 00:57:58.496417 8450 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260924 00:57:59.561275 8450 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260924 00:57:59.561453 8450 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ranger-kms
I20260924 00:57:59.561580 8450 mini_ranger_kms.cc:227] Using postgres at 127.8.64.148:44829
2026-09-24 00:57:59,973 [I] DB FLAVOR :POSTGRES
2026-09-24 00:57:59,973 [I] --------- Verifying Ranger DB connection ---------
2026-09-24 00:57:59,973 [I] Checking connection
2026-09-24 00:57:59,973 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.64.148:44829/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-24 00:58:00,207 [I] connection success
2026-09-24 00:58:00,207 [I] --------- Verifying Ranger DB tables ---------
2026-09-24 00:58:00,208 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-24 00:58:00,208 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.64.148:44829/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-24 00:58:00,443 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-24 00:58:00,444 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-24 00:58:00,444 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-24 00:58:00,444 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.64.148:44829/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-24 00:58:00,716 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260924 00:58:01.219732 8450 mini_ranger_kms.cc:310] Created kms service
I20260924 00:58:01.384533 8450 mini_ranger_kms.cc:326] Created kudu user
I20260924 00:58:01.428385 8450 mini_ranger_kms.cc:343] Created rangerkms user
Sep 24 00:58:01 dist-test-slave-mmhs krb5kdc[19325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211478, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.8.64.148@KRBTEST.COM
I20260924 00:58:01.805779 8450 mini_ranger_kms.cc:384] Added ranger policy
I20260924 00:58:01.806099 8450 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ranger-kms:/tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260924 00:58:01.806155 8450 mini_ranger_kms.cc:240] Using host: 127.8.64.148
I20260924 00:58:01.828119 8450 mini_ranger_kms.cc:276] Ranger KMS PID: 19719
I20260924 00:58:01.828333 8450 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.8.64.148:54711
Sep 24, 2026 12:58:02 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 24, 2026 12:58:02 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 24, 2026 12:58:02 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 24, 2026 12:58:02 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 24, 2026 12:58:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ranger-kms/ews/webapp].
Sep 24, 2026 12:58:03 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Sep 24 00:58:04 dist-test-slave-mmhs krb5kdc[19325](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790211484, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.64.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:58:04.161430 8450 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.689s user 0.114s sys 0.327s
I20260924 00:58:04.161679 8450 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260924 00:58:04.161872 8450 mini_ranger_kms.cc:401] 127.8.64.148:54711/kms/v1/keys
Sep 24 00:58:04 dist-test-slave-mmhs krb5kdc[19325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211484, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.64.148@KRBTEST.COM for HTTP/127.8.64.148@KRBTEST.COM
Sep 24 00:58:04 dist-test-slave-mmhs krb5kdc[19325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211484, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.64.148@KRBTEST.COM for HTTP/127.8.64.148@KRBTEST.COM
Sep 24 00:58:07 dist-test-slave-mmhs krb5kdc[19325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211487, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.8.64.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
I20260924 00:58:07.258800 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:40825
--webserver_interface=127.8.64.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:36245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:40825
--ranger_config_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.64.148:54711/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:58:07.772353 19771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:58:07.772725 19771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:58:07.772910 19771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:58:07.781438 19771 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:58:07.781611 19771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:58:07.781729 19771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:58:07.781864 19771 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:58:07.781970 19771 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:58:07.793408 19771 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:36245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/master-0/wal
--ranger_kms_url=127.8.64.148:54711/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:40825
--ranger_config_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:40825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:58:07.794895 19771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:58:07.796367 19771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:58:07.811625 19776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:58:07.811702 19779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:58:07.811780 19777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:58:07.813134 19771 server_base.cc:1034] running on GCE node
I20260924 00:58:07.813948 19771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:58:07.815511 19771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:58:07.816767 19771 hybrid_clock.cc:648] HybridClock initialized: now 1790211487816699 us; error 40 us; skew 500 ppm
I20260924 00:58:07.817150 19771 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:58:07 dist-test-slave-mmhs krb5kdc[19325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211487, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:58:07.820950 19771 init.cc:377] Logged in from keytab as kudu/127.8.64.190@KRBTEST.COM (short username kudu)
I20260924 00:58:07.822676 19771 webserver.cc:460] Webserver started at http://127.8.64.190:41373/ using document root <none> and password file <none>
I20260924 00:58:07.823388 19771 fs_manager.cc:359] Metadata directory not provided
I20260924 00:58:07.823561 19771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:58:07.823907 19771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 00:58:07 dist-test-slave-mmhs krb5kdc[19325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211487, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for HTTP/127.8.64.148@KRBTEST.COM
I20260924 00:58:07.888402 19771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "78415e9fdfce4775b378a81b340a5bb1"
format_stamp: "Formatted at 2026-09-24 00:58:07 on dist-test-slave-mmhs"
server_key: "cd0dce8b7c670e6b95aaf23464bcd3a2"
server_key_iv: "f8faad2d1a1c3e01799e4a586fa19467"
server_key_version: "kuduclusterkey@0"
I20260924 00:58:07.889328 19771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "78415e9fdfce4775b378a81b340a5bb1"
format_stamp: "Formatted at 2026-09-24 00:58:07 on dist-test-slave-mmhs"
server_key: "cd0dce8b7c670e6b95aaf23464bcd3a2"
server_key_iv: "f8faad2d1a1c3e01799e4a586fa19467"
server_key_version: "kuduclusterkey@0"
I20260924 00:58:07.895447 19771 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.002s
I20260924 00:58:07.913198 19786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:58:07.914948 19771 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260924 00:58:07.915158 19771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "78415e9fdfce4775b378a81b340a5bb1"
format_stamp: "Formatted at 2026-09-24 00:58:07 on dist-test-slave-mmhs"
server_key: "cd0dce8b7c670e6b95aaf23464bcd3a2"
server_key_iv: "f8faad2d1a1c3e01799e4a586fa19467"
server_key_version: "kuduclusterkey@0"
I20260924 00:58:07.915436 19771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:58:07.943635 19771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:58:07.948262 19771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:58:07.948734 19771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:58:07.978820 19771 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.190:40825
I20260924 00:58:07.978880 19837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.190:40825 every 8 connection(s)
I20260924 00:58:07.980329 19771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 00:58:07.983817 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 19771
I20260924 00:58:07.984021 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 00:58:07.984871 8450 external_mini_cluster.cc:1428] Setting key e727e4a1564d2441bf80d81e4e96f988
I20260924 00:58:07.984990 19838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 24 00:58:07 dist-test-slave-mmhs krb5kdc[19325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211487, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:58:08.000314 19838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78415e9fdfce4775b378a81b340a5bb1: Bootstrap starting.
I20260924 00:58:08.004583 19838 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 78415e9fdfce4775b378a81b340a5bb1: Neither blocks nor log segments found. Creating new log.
I20260924 00:58:08.005887 19838 log.cc:824] T 00000000000000000000000000000000 P 78415e9fdfce4775b378a81b340a5bb1: Log is configured to *not* fsync() on all Append() calls
I20260924 00:58:08.009243 19838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78415e9fdfce4775b378a81b340a5bb1: No bootstrap required, opened a new log
I20260924 00:58:08.009748 19841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:58:07.987078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45858 (local address 127.8.64.190:40825)
0924 00:58:07.987966 (+ 888us) server_negotiation.cc:206] Beginning negotiation
0924 00:58:07.987977 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 00:58:07.988026 (+ 49us) server_negotiation.cc:407] Connection header received
0924 00:58:07.990415 (+ 2389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:58:07.990436 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:58:07.990847 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:58:07.991246 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:58:07.992573 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:58:07.994267 (+ 1694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:58:07.995557 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:58:07.996461 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:58:07.996723 (+ 262us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:58:08.001048 (+ 4325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:58:08.001056 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:58:08.001071 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:58:08.001125 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:58:08.005434 (+ 4309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:58:08.006150 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:58:08.006157 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:58:08.006163 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:58:08.006494 (+ 331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:58:08.006910 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:58:08.006915 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:58:08.006922 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:58:08.007457 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:58:08.007781 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0924 00:58:08.008256 (+ 475us) server_negotiation.cc:299] Negotiation successful
0924 00:58:08.008647 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":677,"thread_start_us":468,"threads_started":1}
I20260924 00:58:08.018131 19838 raft_consensus.cc:348] T 00000000000000000000000000000000 P 78415e9fdfce4775b378a81b340a5bb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78415e9fdfce4775b378a81b340a5bb1" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 40825 } }
I20260924 00:58:08.018692 19838 raft_consensus.cc:374] T 00000000000000000000000000000000 P 78415e9fdfce4775b378a81b340a5bb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:58:08.018842 19838 raft_consensus.cc:729] T 00000000000000000000000000000000 P 78415e9fdfce4775b378a81b340a5bb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78415e9fdfce4775b378a81b340a5bb1, State: Initialized, Role: FOLLOWER
I20260924 00:58:08.019562 19838 consensus_queue.cc:260] T 00000000000000000000000000000000 P 78415e9fdfce4775b378a81b340a5bb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78415e9fdfce4775b378a81b340a5bb1" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 40825 } }
I20260924 00:58:08.019840 19838 raft_consensus.cc:388] T 00000000000000000000000000000000 P 78415e9fdfce4775b378a81b340a5bb1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:58:08.019972 19838 raft_consensus.cc:482] T 00000000000000000000000000000000 P 78415e9fdfce4775b378a81b340a5bb1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:58:08.020183 19838 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 78415e9fdfce4775b378a81b340a5bb1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:58:08.022701 19838 raft_consensus.cc:504] T 00000000000000000000000000000000 P 78415e9fdfce4775b378a81b340a5bb1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78415e9fdfce4775b378a81b340a5bb1" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 40825 } }
I20260924 00:58:08.023227 19838 leader_election.cc:302] T 00000000000000000000000000000000 P 78415e9fdfce4775b378a81b340a5bb1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 78415e9fdfce4775b378a81b340a5bb1; no voters:
I20260924 00:58:08.023977 19838 leader_election.cc:288] T 00000000000000000000000000000000 P 78415e9fdfce4775b378a81b340a5bb1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:58:08.024281 19843 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 78415e9fdfce4775b378a81b340a5bb1 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:58:08.025920 19843 raft_consensus.cc:686] T 00000000000000000000000000000000 P 78415e9fdfce4775b378a81b340a5bb1 [term 1 LEADER]: Becoming Leader. State: Replica: 78415e9fdfce4775b378a81b340a5bb1, State: Running, Role: LEADER
I20260924 00:58:08.026714 19843 consensus_queue.cc:237] T 00000000000000000000000000000000 P 78415e9fdfce4775b378a81b340a5bb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78415e9fdfce4775b378a81b340a5bb1" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 40825 } }
I20260924 00:58:08.027024 19838 sys_catalog.cc:564] T 00000000000000000000000000000000 P 78415e9fdfce4775b378a81b340a5bb1 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:58:08.030277 19844 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78415e9fdfce4775b378a81b340a5bb1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "78415e9fdfce4775b378a81b340a5bb1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78415e9fdfce4775b378a81b340a5bb1" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 40825 } } }
I20260924 00:58:08.030334 19845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78415e9fdfce4775b378a81b340a5bb1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78415e9fdfce4775b378a81b340a5bb1. Latest consensus state: current_term: 1 leader_uuid: "78415e9fdfce4775b378a81b340a5bb1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78415e9fdfce4775b378a81b340a5bb1" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 40825 } } }
I20260924 00:58:08.030725 19844 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78415e9fdfce4775b378a81b340a5bb1 [sys.catalog]: This master's current role is: LEADER
I20260924 00:58:08.030779 19845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78415e9fdfce4775b378a81b340a5bb1 [sys.catalog]: This master's current role is: LEADER
I20260924 00:58:08.042407 19849 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:58:08.046952 19849 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:58:08.047770 19838 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260924 00:58:08.065407 19849 catalog_manager.cc:1347] Generated new cluster ID: 944a7210e7a1405fa7153eed7e82295a
I20260924 00:58:08.065742 19849 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:58:08.082995 19849 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:58:08.084393 19849 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:58:08.102626 19849 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 78415e9fdfce4775b378a81b340a5bb1: Generated new TSK 0
I20260924 00:58:08.103591 19849 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.64.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab.
Entry for principal kudu/127.8.64.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab.
I20260924 00:58:08.823184 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.64.148:54711/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.129:0
--local_ip_for_outbound_sockets=127.8.64.129
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:40825
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab
--principal=kudu/127.8.64.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:36245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 00:58:09 dist-test-slave-mmhs krb5kdc[19325](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790211489, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260924 00:58:09.275627 19883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:58:09.276058 19883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:58:09.276285 19883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:58:09.288074 19883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:58:09.288622 19883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:58:09.288887 19883 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.129
I20260924 00:58:09.305402 19883 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:36245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.8.64.148:54711/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab
--principal=kudu/127.8.64.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:40825
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.129
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:58:09.307416 19883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:58:09.309309 19883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:58:09.328234 19893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:58:09.329545 19894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:58:09.338253 19883 server_base.cc:1034] running on GCE node
W20260924 00:58:09.342641 19896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:58:09.343767 19883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:58:09.349949 19883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:58:09.351308 19883 hybrid_clock.cc:648] HybridClock initialized: now 1790211489351208 us; error 91 us; skew 500 ppm
I20260924 00:58:09.351811 19883 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:58:09 dist-test-slave-mmhs krb5kdc[19325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211489, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:58:09.356359 19883 init.cc:377] Logged in from keytab as kudu/127.8.64.129@KRBTEST.COM (short username kudu)
I20260924 00:58:09.358707 19883 webserver.cc:460] Webserver started at http://127.8.64.129:41887/ using document root <none> and password file <none>
I20260924 00:58:09.359840 19883 fs_manager.cc:359] Metadata directory not provided
I20260924 00:58:09.360044 19883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:58:09.360467 19883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 00:58:09 dist-test-slave-mmhs krb5kdc[19325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211489, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for HTTP/127.8.64.148@KRBTEST.COM
I20260924 00:58:09.385913 19883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-0/data/instance:
uuid: "063654ebbb284a1abef76e0bef1aef23"
format_stamp: "Formatted at 2026-09-24 00:58:09 on dist-test-slave-mmhs"
server_key: "177016f05b7d768d675079ea6c824d1d"
server_key_iv: "94dbc71bf81bd2b01946e310f3144477"
server_key_version: "kuduclusterkey@0"
I20260924 00:58:09.387226 19883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance:
uuid: "063654ebbb284a1abef76e0bef1aef23"
format_stamp: "Formatted at 2026-09-24 00:58:09 on dist-test-slave-mmhs"
server_key: "177016f05b7d768d675079ea6c824d1d"
server_key_iv: "94dbc71bf81bd2b01946e310f3144477"
server_key_version: "kuduclusterkey@0"
I20260924 00:58:09.409585 19883 fs_manager.cc:691] Time spent creating directory manager: real 0.021s user 0.014s sys 0.003s
I20260924 00:58:09.443035 19903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:58:09.445329 19883 fs_manager.cc:725] Time spent opening block manager: real 0.017s user 0.012s sys 0.004s
I20260924 00:58:09.445626 19883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-0/wal
uuid: "063654ebbb284a1abef76e0bef1aef23"
format_stamp: "Formatted at 2026-09-24 00:58:09 on dist-test-slave-mmhs"
server_key: "177016f05b7d768d675079ea6c824d1d"
server_key_iv: "94dbc71bf81bd2b01946e310f3144477"
server_key_version: "kuduclusterkey@0"
I20260924 00:58:09.445991 19883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:58:09.470680 19883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:58:09.477397 19883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:58:09.477775 19883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:58:09.502243 19883 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:58:09.510840 19883 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:58:09.511040 19883 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260924 00:58:09.511240 19883 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:58:09.511364 19883 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 00:58:09 dist-test-slave-mmhs krb5kdc[19325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211489, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for HTTP/127.8.64.148@KRBTEST.COM
I20260924 00:58:09.967875 19883 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.129:39865
I20260924 00:58:09.967901 20016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.129:39865 every 8 connection(s)
I20260924 00:58:09.969769 19883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
I20260924 00:58:09.973743 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 19883
I20260924 00:58:09.973964 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance
I20260924 00:58:09.975015 8450 external_mini_cluster.cc:1428] Setting key 3d5a3cda71575ca74d7a53c046a86737
Sep 24 00:58:10 dist-test-slave-mmhs krb5kdc[19325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211489, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.64.130@KRBTEST.COM; defaulting to no policy
I20260924 00:58:10.017467 20019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:58:09.972553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:37165 (local address 127.8.64.190:40825)
0924 00:58:09.994881 (+ 22328us) server_negotiation.cc:206] Beginning negotiation
0924 00:58:09.994888 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:58:09.994915 (+ 27us) server_negotiation.cc:407] Connection header received
0924 00:58:09.994982 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:58:09.994987 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:58:09.995069 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:58:09.997897 (+ 2828us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:58:09.998072 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:58:09.999217 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:58:10.001666 (+ 2449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:58:10.002245 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:58:10.006208 (+ 3963us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:58:10.008129 (+ 1921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:58:10.008137 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:58:10.008142 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:58:10.008186 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:58:10.012372 (+ 4186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:58:10.013497 (+ 1125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:58:10.013503 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:58:10.013507 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:58:10.013671 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:58:10.014268 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:58:10.014272 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:58:10.014275 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:58:10.014654 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:58:10.016732 (+ 2078us) server_negotiation.cc:1035] Waiting for connection context
0924 00:58:10.016875 (+ 143us) server_negotiation.cc:299] Negotiation successful
0924 00:58:10.017100 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22186,"thread_start_us":22016,"threads_started":1}
I20260924 00:58:10.017750 20020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:58:09.973141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:40825 (local address 127.8.64.129:37165)
0924 00:58:09.984696 (+ 11555us) negotiation.cc:107] Waiting for socket to connect
0924 00:58:09.984769 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0924 00:58:09.989038 (+ 4269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:58:09.995773 (+ 6735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:58:09.995789 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:58:09.996705 (+ 916us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:58:09.997647 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:58:09.997687 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:58:09.999426 (+ 1739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:58:09.999433 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:58:10.001447 (+ 2014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:58:10.001477 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:58:10.003298 (+ 1821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:58:10.003304 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:58:10.003581 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:58:10.004422 (+ 841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:58:10.004452 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:58:10.007797 (+ 3345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:58:10.012579 (+ 4782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:58:10.012591 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:58:10.012612 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:58:10.013308 (+ 696us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:58:10.013827 (+ 519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:58:10.013833 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:58:10.013836 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:58:10.014107 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:58:10.014838 (+ 731us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:58:10.014847 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:58:10.015295 (+ 448us) client_negotiation.cc:769] Sending connection context
0924 00:58:10.015624 (+ 329us) client_negotiation.cc:240] Negotiation successful
0924 00:58:10.016196 (+ 572us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11144,"thread_start_us":13364,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.130@KRBTEST.COM" created.
I20260924 00:58:10.020015 20017 heartbeater.cc:347] Connected to a master server at 127.8.64.190:40825
I20260924 00:58:10.020362 20017 heartbeater.cc:464] Registering TS with master...
I20260924 00:58:10.021247 20017 heartbeater.cc:511] Master 127.8.64.190:40825 requested a full tablet report, sending...
I20260924 00:58:10.023614 19803 ts_manager.cc:194] Registered new tserver with Master: 063654ebbb284a1abef76e0bef1aef23 (127.8.64.129:39865)
I20260924 00:58:10.025527 19803 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.129@KRBTEST.COM'} at 127.8.64.129:37165
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab.
Entry for principal kudu/127.8.64.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab.
I20260924 00:58:10.058758 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.64.148:54711/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.130:0
--local_ip_for_outbound_sockets=127.8.64.130
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:40825
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab
--principal=kudu/127.8.64.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:36245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 00:58:10 dist-test-slave-mmhs krb5kdc[19325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211489, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for HTTP/127.8.64.148@KRBTEST.COM
W20260924 00:58:10.503435 20023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:58:10.503724 20023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:58:10.503917 20023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:58:10.511263 20023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:58:10.511379 20023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:58:10.511561 20023 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.130
I20260924 00:58:10.522082 20023 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:36245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.8.64.148:54711/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab
--principal=kudu/127.8.64.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:40825
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.130
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:58:10.523577 20023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:58:10.524892 20023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:58:10.538388 20037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:58:10.538463 20038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:58:10.538419 20040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:58:10.544262 20023 server_base.cc:1034] running on GCE node
I20260924 00:58:10.544945 20023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:58:10.545991 20023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:58:10.547220 20023 hybrid_clock.cc:648] HybridClock initialized: now 1790211490547153 us; error 70 us; skew 500 ppm
I20260924 00:58:10.547636 20023 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:58:10 dist-test-slave-mmhs krb5kdc[19325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211490, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:58:10.556777 20023 init.cc:377] Logged in from keytab as kudu/127.8.64.130@KRBTEST.COM (short username kudu)
I20260924 00:58:10.558707 20023 webserver.cc:460] Webserver started at http://127.8.64.130:33489/ using document root <none> and password file <none>
I20260924 00:58:10.559541 20023 fs_manager.cc:359] Metadata directory not provided
I20260924 00:58:10.559726 20023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:58:10.560134 20023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 00:58:10 dist-test-slave-mmhs krb5kdc[19325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211490, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.130@KRBTEST.COM for HTTP/127.8.64.148@KRBTEST.COM
I20260924 00:58:10.575963 20023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-1/data/instance:
uuid: "522055383d734abf9e6190e1f05156cc"
format_stamp: "Formatted at 2026-09-24 00:58:10 on dist-test-slave-mmhs"
server_key: "1f70a2c1c83dbd40dfb325ca04fc4056"
server_key_iv: "8bb6e793394f074181a03272652a64bf"
server_key_version: "kuduclusterkey@0"
I20260924 00:58:10.577205 20023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance:
uuid: "522055383d734abf9e6190e1f05156cc"
format_stamp: "Formatted at 2026-09-24 00:58:10 on dist-test-slave-mmhs"
server_key: "1f70a2c1c83dbd40dfb325ca04fc4056"
server_key_iv: "8bb6e793394f074181a03272652a64bf"
server_key_version: "kuduclusterkey@0"
I20260924 00:58:10.584576 20023 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260924 00:58:10.603060 20047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:58:10.604866 20023 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.003s
I20260924 00:58:10.605064 20023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-1/wal
uuid: "522055383d734abf9e6190e1f05156cc"
format_stamp: "Formatted at 2026-09-24 00:58:10 on dist-test-slave-mmhs"
server_key: "1f70a2c1c83dbd40dfb325ca04fc4056"
server_key_iv: "8bb6e793394f074181a03272652a64bf"
server_key_version: "kuduclusterkey@0"
I20260924 00:58:10.605329 20023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:58:10.627034 20023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:58:10.631676 20023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:58:10.632051 20023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:58:10.633040 20023 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:58:10.634481 20023 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:58:10.634553 20023 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:58:10.634792 20023 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:58:10.634848 20023 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:58:10.691319 20023 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.130:43303
I20260924 00:58:10.691457 20160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.130:43303 every 8 connection(s)
I20260924 00:58:10.692677 20023 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
I20260924 00:58:10.693259 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 20023
I20260924 00:58:10.693463 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance
I20260924 00:58:10.694156 8450 external_mini_cluster.cc:1428] Setting key 355a88ebe217976af5990fe02ed66a7c
Sep 24 00:58:10 dist-test-slave-mmhs krb5kdc[19325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211490, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.130@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:58:10.715186 20163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:58:10.695571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:45317 (local address 127.8.64.190:40825)
0924 00:58:10.696062 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0924 00:58:10.696068 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:58:10.697306 (+ 1238us) server_negotiation.cc:407] Connection header received
0924 00:58:10.699197 (+ 1891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:58:10.699203 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:58:10.699300 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:58:10.699415 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:58:10.700789 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:58:10.701554 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:58:10.703188 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:58:10.703593 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:58:10.703728 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:58:10.707328 (+ 3600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:58:10.707337 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:58:10.707345 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:58:10.707391 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:58:10.711648 (+ 4257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:58:10.712495 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:58:10.712503 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:58:10.712518 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:58:10.712684 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:58:10.713112 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:58:10.713120 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:58:10.713126 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:58:10.713533 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:58:10.713696 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0924 00:58:10.714388 (+ 692us) server_negotiation.cc:299] Negotiation successful
0924 00:58:10.714690 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":253,"threads_started":1}
I20260924 00:58:10.715876 20164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:58:10.696159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:40825 (local address 127.8.64.130:45317)
0924 00:58:10.697159 (+ 1000us) negotiation.cc:107] Waiting for socket to connect
0924 00:58:10.697195 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 00:58:10.698938 (+ 1743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:58:10.699642 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:58:10.699650 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:58:10.700067 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:58:10.700628 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:58:10.700644 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:58:10.701715 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:58:10.701720 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:58:10.703033 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:58:10.703052 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:58:10.703814 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:58:10.703821 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:58:10.704055 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:58:10.704836 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:58:10.704859 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:58:10.707094 (+ 2235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:58:10.711847 (+ 4753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:58:10.711858 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:58:10.711881 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:58:10.712353 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:58:10.712799 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:58:10.712803 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:58:10.712805 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:58:10.712994 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:58:10.713663 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:58:10.713669 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:58:10.713968 (+ 299us) client_negotiation.cc:769] Sending connection context
0924 00:58:10.714207 (+ 239us) client_negotiation.cc:240] Negotiation successful
0924 00:58:10.714680 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":747,"thread_start_us":544,"threads_started":1}
I20260924 00:58:10.717338 20161 heartbeater.cc:347] Connected to a master server at 127.8.64.190:40825
I20260924 00:58:10.717708 20161 heartbeater.cc:464] Registering TS with master...
I20260924 00:58:10.718688 20161 heartbeater.cc:511] Master 127.8.64.190:40825 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.64.131@KRBTEST.COM; defaulting to no policy
I20260924 00:58:10.720454 19803 ts_manager.cc:194] Registered new tserver with Master: 522055383d734abf9e6190e1f05156cc (127.8.64.130:43303)
I20260924 00:58:10.721290 19803 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.130@KRBTEST.COM'} at 127.8.64.130:45317
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab.
Entry for principal kudu/127.8.64.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab.
I20260924 00:58:10.751928 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.64.148:54711/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.131:0
--local_ip_for_outbound_sockets=127.8.64.131
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:40825
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab
--principal=kudu/127.8.64.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:36245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 00:58:11.029405 20017 heartbeater.cc:503] Master 127.8.64.190:40825 was elected leader, sending a full tablet report...
W20260924 00:58:11.105325 20167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:58:11.105638 20167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:58:11.105810 20167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:58:11.114723 20167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:58:11.114845 20167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:58:11.115036 20167 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.131
I20260924 00:58:11.125353 20167 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:36245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.8.64.148:54711/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab
--principal=kudu/127.8.64.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:40825
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.131
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:58:11.126778 20167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:58:11.128218 20167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:58:11.142153 20173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:58:11.142246 20175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:58:11.142155 20172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:58:11.143002 20167 server_base.cc:1034] running on GCE node
I20260924 00:58:11.143610 20167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:58:11.144539 20167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:58:11.145834 20167 hybrid_clock.cc:648] HybridClock initialized: now 1790211491145754 us; error 66 us; skew 500 ppm
I20260924 00:58:11.146203 20167 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:58:11 dist-test-slave-mmhs krb5kdc[19325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211491, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:58:11.149420 20167 init.cc:377] Logged in from keytab as kudu/127.8.64.131@KRBTEST.COM (short username kudu)
I20260924 00:58:11.150890 20167 webserver.cc:460] Webserver started at http://127.8.64.131:46347/ using document root <none> and password file <none>
I20260924 00:58:11.151602 20167 fs_manager.cc:359] Metadata directory not provided
I20260924 00:58:11.151701 20167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:58:11.151958 20167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 00:58:11 dist-test-slave-mmhs krb5kdc[19325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211491, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.131@KRBTEST.COM for HTTP/127.8.64.148@KRBTEST.COM
I20260924 00:58:11.169862 20167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-2/data/instance:
uuid: "848888dffcd84201b00f8077992b6c99"
format_stamp: "Formatted at 2026-09-24 00:58:11 on dist-test-slave-mmhs"
server_key: "7d646f0842c03a53a25318ff36d05ff8"
server_key_iv: "fcf302ded1e57814272349e728eb03e8"
server_key_version: "kuduclusterkey@0"
I20260924 00:58:11.171042 20167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance:
uuid: "848888dffcd84201b00f8077992b6c99"
format_stamp: "Formatted at 2026-09-24 00:58:11 on dist-test-slave-mmhs"
server_key: "7d646f0842c03a53a25318ff36d05ff8"
server_key_iv: "fcf302ded1e57814272349e728eb03e8"
server_key_version: "kuduclusterkey@0"
I20260924 00:58:11.178426 20167 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260924 00:58:11.197803 20182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:58:11.199438 20167 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260924 00:58:11.199637 20167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-2/wal
uuid: "848888dffcd84201b00f8077992b6c99"
format_stamp: "Formatted at 2026-09-24 00:58:11 on dist-test-slave-mmhs"
server_key: "7d646f0842c03a53a25318ff36d05ff8"
server_key_iv: "fcf302ded1e57814272349e728eb03e8"
server_key_version: "kuduclusterkey@0"
I20260924 00:58:11.199873 20167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:58:11.224328 20167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:58:11.228991 20167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:58:11.229333 20167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:58:11.230422 20167 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:58:11.231938 20167 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:58:11.232044 20167 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:58:11.232190 20167 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:58:11.232270 20167 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:58:11.281929 20167 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.131:36607
I20260924 00:58:11.282022 20294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.131:36607 every 8 connection(s)
I20260924 00:58:11.283327 20167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
I20260924 00:58:11.287180 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 20167
I20260924 00:58:11.287374 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance
I20260924 00:58:11.287966 8450 external_mini_cluster.cc:1428] Setting key 574e452268ea1079887932d51cfa75d2
Sep 24 00:58:11 dist-test-slave-mmhs krb5kdc[19325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211491, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.131@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:58:11.313747 20298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:58:11.288287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:59911 (local address 127.8.64.190:40825)
0924 00:58:11.289250 (+ 963us) server_negotiation.cc:206] Beginning negotiation
0924 00:58:11.289262 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0924 00:58:11.295227 (+ 5965us) server_negotiation.cc:407] Connection header received
0924 00:58:11.297313 (+ 2086us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:58:11.297319 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:58:11.297444 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:58:11.297589 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:58:11.299353 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:58:11.300136 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:58:11.301797 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:58:11.302250 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:58:11.302438 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:58:11.306992 (+ 4554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:58:11.306997 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:58:11.307000 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:58:11.307034 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:58:11.310360 (+ 3326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:58:11.311346 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:58:11.311353 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:58:11.311357 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:58:11.311498 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:58:11.311999 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:58:11.312003 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:58:11.312005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:58:11.312270 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:58:11.312394 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0924 00:58:11.313295 (+ 901us) server_negotiation.cc:299] Negotiation successful
0924 00:58:11.313477 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":789,"thread_start_us":640,"threads_started":1}
I20260924 00:58:11.315140 20297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:58:11.287729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:40825 (local address 127.8.64.131:59911)
0924 00:58:11.295039 (+ 7310us) negotiation.cc:107] Waiting for socket to connect
0924 00:58:11.295105 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0924 00:58:11.297007 (+ 1902us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:58:11.297856 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:58:11.297866 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:58:11.298526 (+ 660us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:58:11.299162 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:58:11.299181 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:58:11.300313 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:58:11.300319 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:58:11.301625 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:58:11.301638 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:58:11.302541 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:58:11.302550 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:58:11.302828 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:58:11.303918 (+ 1090us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:58:11.303954 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:58:11.306742 (+ 2788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:58:11.310533 (+ 3791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:58:11.310544 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:58:11.310563 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:58:11.311195 (+ 632us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:58:11.311603 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:58:11.311608 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:58:11.311612 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:58:11.311882 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:58:11.312398 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:58:11.312408 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:58:11.312840 (+ 432us) client_negotiation.cc:769] Sending connection context
0924 00:58:11.313159 (+ 319us) client_negotiation.cc:240] Negotiation successful
0924 00:58:11.313719 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6985,"thread_start_us":6744,"threads_started":1}
I20260924 00:58:11.317019 20295 heartbeater.cc:347] Connected to a master server at 127.8.64.190:40825
I20260924 00:58:11.317353 20295 heartbeater.cc:464] Registering TS with master...
I20260924 00:58:11.318264 20295 heartbeater.cc:511] Master 127.8.64.190:40825 requested a full tablet report, sending...
I20260924 00:58:11.320106 19802 ts_manager.cc:194] Registered new tserver with Master: 848888dffcd84201b00f8077992b6c99 (127.8.64.131:36607)
I20260924 00:58:11.321352 19802 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.131@KRBTEST.COM'} at 127.8.64.131:59911
I20260924 00:58:11.330564 8450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 00:58:11.351759 20298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:58:11.336395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45868 (local address 127.8.64.190:40825)
0924 00:58:11.336607 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0924 00:58:11.336613 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:58:11.337184 (+ 571us) server_negotiation.cc:407] Connection header received
0924 00:58:11.337340 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:58:11.337345 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:58:11.337442 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:58:11.337561 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:58:11.339131 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:58:11.340233 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:58:11.341577 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:58:11.342219 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:58:11.342375 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:58:11.345063 (+ 2688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:58:11.345069 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:58:11.345074 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:58:11.345114 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:58:11.348836 (+ 3722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:58:11.349508 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:58:11.349513 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:58:11.349518 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:58:11.349673 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:58:11.350043 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:58:11.350048 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:58:11.350052 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:58:11.350445 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:58:11.350694 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0924 00:58:11.351063 (+ 369us) server_negotiation.cc:299] Negotiation successful
0924 00:58:11.351384 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260924 00:58:11.356302 19802 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45868:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:58:11.363785 19802 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:58:11.404450 20309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:58:11.377709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.130:43303 (local address 127.0.0.1:59928)
0924 00:58:11.392034 (+ 14325us) negotiation.cc:107] Waiting for socket to connect
0924 00:58:11.392102 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0924 00:58:11.392328 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:58:11.394709 (+ 2381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:58:11.394720 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:58:11.394763 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:58:11.395268 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:58:11.395284 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:58:11.397867 (+ 2583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:58:11.397875 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:58:11.401375 (+ 3500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:58:11.401389 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:58:11.403478 (+ 2089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:58:11.403485 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:58:11.403672 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:58:11.403719 (+ 47us) client_negotiation.cc:769] Sending connection context
0924 00:58:11.403841 (+ 122us) client_negotiation.cc:240] Negotiation successful
0924 00:58:11.404026 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13962,"thread_start_us":15172,"threads_started":1}
I20260924 00:58:11.405196 20310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:58:11.378049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59928 (local address 127.8.64.130:43303)
0924 00:58:11.393610 (+ 15561us) server_negotiation.cc:206] Beginning negotiation
0924 00:58:11.393621 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 00:58:11.393651 (+ 30us) server_negotiation.cc:407] Connection header received
0924 00:58:11.393744 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:58:11.393751 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:58:11.394116 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:58:11.394362 (+ 246us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:58:11.395521 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:58:11.397666 (+ 2145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:58:11.401564 (+ 3898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:58:11.403005 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:58:11.403132 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:58:11.403341 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0924 00:58:11.404721 (+ 1380us) server_negotiation.cc:299] Negotiation successful
0924 00:58:11.404892 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15382,"thread_start_us":15276,"threads_started":1}
I20260924 00:58:11.409950 20308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:58:11.377946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:39865 (local address 127.0.0.1:50636)
0924 00:58:11.392717 (+ 14771us) negotiation.cc:107] Waiting for socket to connect
0924 00:58:11.392736 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0924 00:58:11.392830 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:58:11.397250 (+ 4420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:58:11.397254 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:58:11.397280 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:58:11.397658 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:58:11.397671 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:58:11.399610 (+ 1939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:58:11.399615 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:58:11.403764 (+ 4149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:58:11.403779 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:58:11.406156 (+ 2377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:58:11.406162 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:58:11.406391 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:58:11.409322 (+ 2931us) client_negotiation.cc:769] Sending connection context
0924 00:58:11.409411 (+ 89us) client_negotiation.cc:240] Negotiation successful
0924 00:58:11.409593 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14658,"mutex_wait_us":5,"thread_start_us":14633,"threads_started":1}
I20260924 00:58:11.410018 20312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:58:11.393672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52020 (local address 127.8.64.131:36607)
0924 00:58:11.400536 (+ 6864us) server_negotiation.cc:206] Beginning negotiation
0924 00:58:11.400548 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0924 00:58:11.401397 (+ 849us) server_negotiation.cc:407] Connection header received
0924 00:58:11.401506 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:58:11.401514 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:58:11.401786 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:58:11.401980 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:58:11.403096 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:58:11.404972 (+ 1876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:58:11.407538 (+ 2566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:58:11.408927 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:58:11.409064 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:58:11.409265 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0924 00:58:11.409546 (+ 281us) server_negotiation.cc:299] Negotiation successful
0924 00:58:11.409707 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6691,"thread_start_us":6549,"threads_started":1}
I20260924 00:58:11.409468 20095 tablet_service.cc:1467] Processing CreateTablet for tablet e1258d0920594663bb850478418ddc63 (DEFAULT_TABLE table=test-table [id=c8496279cdac414facd168dc91b73284]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:58:11.410904 20313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:58:11.399681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.131:36607 (local address 127.0.0.1:52020)
0924 00:58:11.401122 (+ 1441us) negotiation.cc:107] Waiting for socket to connect
0924 00:58:11.401140 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0924 00:58:11.401257 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:58:11.402235 (+ 978us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:58:11.402240 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:58:11.402268 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:58:11.402884 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:58:11.402896 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:58:11.405133 (+ 2237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:58:11.405139 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:58:11.407333 (+ 2194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:58:11.407347 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:58:11.409143 (+ 1796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:58:11.409150 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:58:11.409318 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:58:11.409349 (+ 31us) client_negotiation.cc:769] Sending connection context
0924 00:58:11.409425 (+ 76us) client_negotiation.cc:240] Negotiation successful
0924 00:58:11.410520 (+ 1095us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1303,"mutex_wait_us":958,"thread_start_us":2630,"threads_started":1}
I20260924 00:58:11.411249 20095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1258d0920594663bb850478418ddc63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:58:11.412257 20311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:58:11.378247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50636 (local address 127.8.64.129:39865)
0924 00:58:11.396734 (+ 18487us) server_negotiation.cc:206] Beginning negotiation
0924 00:58:11.396742 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:58:11.396767 (+ 25us) server_negotiation.cc:407] Connection header received
0924 00:58:11.396849 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:58:11.396855 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:58:11.397040 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:58:11.397254 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:58:11.397856 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:58:11.399440 (+ 1584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:58:11.403949 (+ 4509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:58:11.405657 (+ 1708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:58:11.405777 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:58:11.406004 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0924 00:58:11.411236 (+ 5232us) server_negotiation.cc:299] Negotiation successful
0924 00:58:11.411413 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18324,"thread_start_us":19189,"threads_started":1}
I20260924 00:58:11.416519 19952 tablet_service.cc:1467] Processing CreateTablet for tablet e1258d0920594663bb850478418ddc63 (DEFAULT_TABLE table=test-table [id=c8496279cdac414facd168dc91b73284]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:58:11.416986 20230 tablet_service.cc:1467] Processing CreateTablet for tablet e1258d0920594663bb850478418ddc63 (DEFAULT_TABLE table=test-table [id=c8496279cdac414facd168dc91b73284]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:58:11.418462 20230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1258d0920594663bb850478418ddc63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:58:11.419302 19952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1258d0920594663bb850478418ddc63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:58:11.432586 20314 tablet_bootstrap.cc:492] T e1258d0920594663bb850478418ddc63 P 522055383d734abf9e6190e1f05156cc: Bootstrap starting.
I20260924 00:58:11.439587 20314 tablet_bootstrap.cc:654] T e1258d0920594663bb850478418ddc63 P 522055383d734abf9e6190e1f05156cc: Neither blocks nor log segments found. Creating new log.
I20260924 00:58:11.441601 20314 log.cc:824] T e1258d0920594663bb850478418ddc63 P 522055383d734abf9e6190e1f05156cc: Log is configured to *not* fsync() on all Append() calls
I20260924 00:58:11.444166 20316 tablet_bootstrap.cc:492] T e1258d0920594663bb850478418ddc63 P 063654ebbb284a1abef76e0bef1aef23: Bootstrap starting.
I20260924 00:58:11.449326 20316 tablet_bootstrap.cc:654] T e1258d0920594663bb850478418ddc63 P 063654ebbb284a1abef76e0bef1aef23: Neither blocks nor log segments found. Creating new log.
I20260924 00:58:11.441700 20315 tablet_bootstrap.cc:492] T e1258d0920594663bb850478418ddc63 P 848888dffcd84201b00f8077992b6c99: Bootstrap starting.
I20260924 00:58:11.451120 20316 log.cc:824] T e1258d0920594663bb850478418ddc63 P 063654ebbb284a1abef76e0bef1aef23: Log is configured to *not* fsync() on all Append() calls
I20260924 00:58:11.454175 20315 tablet_bootstrap.cc:654] T e1258d0920594663bb850478418ddc63 P 848888dffcd84201b00f8077992b6c99: Neither blocks nor log segments found. Creating new log.
I20260924 00:58:11.454401 20314 tablet_bootstrap.cc:492] T e1258d0920594663bb850478418ddc63 P 522055383d734abf9e6190e1f05156cc: No bootstrap required, opened a new log
I20260924 00:58:11.455078 20314 ts_tablet_manager.cc:1397] T e1258d0920594663bb850478418ddc63 P 522055383d734abf9e6190e1f05156cc: Time spent bootstrapping tablet: real 0.022s user 0.015s sys 0.004s
I20260924 00:58:11.455155 20316 tablet_bootstrap.cc:492] T e1258d0920594663bb850478418ddc63 P 063654ebbb284a1abef76e0bef1aef23: No bootstrap required, opened a new log
I20260924 00:58:11.455411 20316 ts_tablet_manager.cc:1397] T e1258d0920594663bb850478418ddc63 P 063654ebbb284a1abef76e0bef1aef23: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260924 00:58:11.455906 20315 log.cc:824] T e1258d0920594663bb850478418ddc63 P 848888dffcd84201b00f8077992b6c99: Log is configured to *not* fsync() on all Append() calls
I20260924 00:58:11.465466 20315 tablet_bootstrap.cc:492] T e1258d0920594663bb850478418ddc63 P 848888dffcd84201b00f8077992b6c99: No bootstrap required, opened a new log
I20260924 00:58:11.465842 20315 ts_tablet_manager.cc:1397] T e1258d0920594663bb850478418ddc63 P 848888dffcd84201b00f8077992b6c99: Time spent bootstrapping tablet: real 0.024s user 0.007s sys 0.008s
I20260924 00:58:11.467522 20314 raft_consensus.cc:348] T e1258d0920594663bb850478418ddc63 P 522055383d734abf9e6190e1f05156cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522055383d734abf9e6190e1f05156cc" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 43303 } } peers { permanent_uuid: "063654ebbb284a1abef76e0bef1aef23" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 39865 } } peers { permanent_uuid: "848888dffcd84201b00f8077992b6c99" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36607 } }
I20260924 00:58:11.468106 20314 raft_consensus.cc:374] T e1258d0920594663bb850478418ddc63 P 522055383d734abf9e6190e1f05156cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:58:11.467759 20316 raft_consensus.cc:348] T e1258d0920594663bb850478418ddc63 P 063654ebbb284a1abef76e0bef1aef23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522055383d734abf9e6190e1f05156cc" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 43303 } } peers { permanent_uuid: "063654ebbb284a1abef76e0bef1aef23" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 39865 } } peers { permanent_uuid: "848888dffcd84201b00f8077992b6c99" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36607 } }
I20260924 00:58:11.468245 20314 raft_consensus.cc:729] T e1258d0920594663bb850478418ddc63 P 522055383d734abf9e6190e1f05156cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 522055383d734abf9e6190e1f05156cc, State: Initialized, Role: FOLLOWER
I20260924 00:58:11.468344 20316 raft_consensus.cc:374] T e1258d0920594663bb850478418ddc63 P 063654ebbb284a1abef76e0bef1aef23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:58:11.468479 20316 raft_consensus.cc:729] T e1258d0920594663bb850478418ddc63 P 063654ebbb284a1abef76e0bef1aef23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 063654ebbb284a1abef76e0bef1aef23, State: Initialized, Role: FOLLOWER
I20260924 00:58:11.469095 20314 consensus_queue.cc:260] T e1258d0920594663bb850478418ddc63 P 522055383d734abf9e6190e1f05156cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522055383d734abf9e6190e1f05156cc" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 43303 } } peers { permanent_uuid: "063654ebbb284a1abef76e0bef1aef23" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 39865 } } peers { permanent_uuid: "848888dffcd84201b00f8077992b6c99" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36607 } }
I20260924 00:58:11.469151 20316 consensus_queue.cc:260] T e1258d0920594663bb850478418ddc63 P 063654ebbb284a1abef76e0bef1aef23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522055383d734abf9e6190e1f05156cc" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 43303 } } peers { permanent_uuid: "063654ebbb284a1abef76e0bef1aef23" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 39865 } } peers { permanent_uuid: "848888dffcd84201b00f8077992b6c99" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36607 } }
I20260924 00:58:11.477411 20314 ts_tablet_manager.cc:1428] T e1258d0920594663bb850478418ddc63 P 522055383d734abf9e6190e1f05156cc: Time spent starting tablet: real 0.022s user 0.010s sys 0.010s
I20260924 00:58:11.477833 20161 heartbeater.cc:503] Master 127.8.64.190:40825 was elected leader, sending a full tablet report...
I20260924 00:58:11.478379 20315 raft_consensus.cc:348] T e1258d0920594663bb850478418ddc63 P 848888dffcd84201b00f8077992b6c99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522055383d734abf9e6190e1f05156cc" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 43303 } } peers { permanent_uuid: "063654ebbb284a1abef76e0bef1aef23" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 39865 } } peers { permanent_uuid: "848888dffcd84201b00f8077992b6c99" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36607 } }
I20260924 00:58:11.478819 20315 raft_consensus.cc:374] T e1258d0920594663bb850478418ddc63 P 848888dffcd84201b00f8077992b6c99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:58:11.478905 20315 raft_consensus.cc:729] T e1258d0920594663bb850478418ddc63 P 848888dffcd84201b00f8077992b6c99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 848888dffcd84201b00f8077992b6c99, State: Initialized, Role: FOLLOWER
I20260924 00:58:11.480983 20316 ts_tablet_manager.cc:1428] T e1258d0920594663bb850478418ddc63 P 063654ebbb284a1abef76e0bef1aef23: Time spent starting tablet: real 0.025s user 0.015s sys 0.007s
I20260924 00:58:11.481921 20315 consensus_queue.cc:260] T e1258d0920594663bb850478418ddc63 P 848888dffcd84201b00f8077992b6c99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522055383d734abf9e6190e1f05156cc" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 43303 } } peers { permanent_uuid: "063654ebbb284a1abef76e0bef1aef23" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 39865 } } peers { permanent_uuid: "848888dffcd84201b00f8077992b6c99" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36607 } }
I20260924 00:58:11.485375 20295 heartbeater.cc:503] Master 127.8.64.190:40825 was elected leader, sending a full tablet report...
I20260924 00:58:11.486050 20315 ts_tablet_manager.cc:1428] T e1258d0920594663bb850478418ddc63 P 848888dffcd84201b00f8077992b6c99: Time spent starting tablet: real 0.020s user 0.016s sys 0.000s
W20260924 00:58:11.504581 20018 tablet.cc:2369] T e1258d0920594663bb850478418ddc63 P 063654ebbb284a1abef76e0bef1aef23: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 00:58:11.535542 20296 tablet.cc:2369] T e1258d0920594663bb850478418ddc63 P 848888dffcd84201b00f8077992b6c99: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 00:58:11.697522 20162 tablet.cc:2369] T e1258d0920594663bb850478418ddc63 P 522055383d734abf9e6190e1f05156cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:58:11.823577 20322 raft_consensus.cc:482] T e1258d0920594663bb850478418ddc63 P 848888dffcd84201b00f8077992b6c99 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:58:11.823916 20322 raft_consensus.cc:504] T e1258d0920594663bb850478418ddc63 P 848888dffcd84201b00f8077992b6c99 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522055383d734abf9e6190e1f05156cc" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 43303 } } peers { permanent_uuid: "063654ebbb284a1abef76e0bef1aef23" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 39865 } } peers { permanent_uuid: "848888dffcd84201b00f8077992b6c99" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36607 } }
I20260924 00:58:11.825538 20322 leader_election.cc:288] T e1258d0920594663bb850478418ddc63 P 848888dffcd84201b00f8077992b6c99 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 522055383d734abf9e6190e1f05156cc (127.8.64.130:43303), 063654ebbb284a1abef76e0bef1aef23 (127.8.64.129:39865)
I20260924 00:58:11.831441 20320 raft_consensus.cc:482] T e1258d0920594663bb850478418ddc63 P 063654ebbb284a1abef76e0bef1aef23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:58:11.831807 20320 raft_consensus.cc:504] T e1258d0920594663bb850478418ddc63 P 063654ebbb284a1abef76e0bef1aef23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522055383d734abf9e6190e1f05156cc" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 43303 } } peers { permanent_uuid: "063654ebbb284a1abef76e0bef1aef23" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 39865 } } peers { permanent_uuid: "848888dffcd84201b00f8077992b6c99" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36607 } }
I20260924 00:58:11.833600 20310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:58:11.825945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:41433 (local address 127.8.64.130:43303)
0924 00:58:11.826226 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0924 00:58:11.826235 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:58:11.827189 (+ 954us) server_negotiation.cc:407] Connection header received
0924 00:58:11.827411 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:58:11.827416 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:58:11.827524 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:58:11.827624 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:58:11.828718 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:58:11.829409 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:58:11.831377 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:58:11.832547 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:58:11.832730 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:58:11.832833 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0924 00:58:11.833221 (+ 388us) server_negotiation.cc:299] Negotiation successful
0924 00:58:11.833346 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260924 00:58:11.833758 20311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:58:11.826059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:38561 (local address 127.8.64.129:39865)
0924 00:58:11.826327 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0924 00:58:11.826335 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:58:11.827248 (+ 913us) server_negotiation.cc:407] Connection header received
0924 00:58:11.827471 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:58:11.827475 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:58:11.827531 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:58:11.827630 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:58:11.828615 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:58:11.829313 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:58:11.831250 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:58:11.832514 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:58:11.832708 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:58:11.832786 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0924 00:58:11.833385 (+ 599us) server_negotiation.cc:299] Negotiation successful
0924 00:58:11.833522 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260924 00:58:11.833843 20324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:58:11.825801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.130:43303 (local address 127.8.64.131:41433)
0924 00:58:11.827067 (+ 1266us) negotiation.cc:107] Waiting for socket to connect
0924 00:58:11.827095 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0924 00:58:11.827278 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:58:11.827639 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:58:11.827645 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:58:11.827667 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:58:11.828545 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:58:11.828561 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:58:11.829548 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:58:11.829554 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:58:11.831196 (+ 1642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:58:11.831210 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:58:11.832718 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:58:11.832724 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:58:11.832933 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:58:11.833065 (+ 132us) client_negotiation.cc:769] Sending connection context
0924 00:58:11.833166 (+ 101us) client_negotiation.cc:240] Negotiation successful
0924 00:58:11.833356 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1144,"mutex_wait_us":36,"thread_start_us":1069,"threads_started":1}
I20260924 00:58:11.833968 20320 leader_election.cc:288] T e1258d0920594663bb850478418ddc63 P 063654ebbb284a1abef76e0bef1aef23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 522055383d734abf9e6190e1f05156cc (127.8.64.130:43303), 848888dffcd84201b00f8077992b6c99 (127.8.64.131:36607)
I20260924 00:58:11.834579 20323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:58:11.825903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:39865 (local address 127.8.64.131:38561)
0924 00:58:11.827139 (+ 1236us) negotiation.cc:107] Waiting for socket to connect
0924 00:58:11.827154 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 00:58:11.827310 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:58:11.827678 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:58:11.827683 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:58:11.827705 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:58:11.828428 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:58:11.828440 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:58:11.829453 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:58:11.829461 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:58:11.831030 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:58:11.831070 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:58:11.832928 (+ 1858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:58:11.832934 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:58:11.833216 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:58:11.833245 (+ 29us) client_negotiation.cc:769] Sending connection context
0924 00:58:11.834009 (+ 764us) client_negotiation.cc:240] Negotiation successful
0924 00:58:11.834160 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1150,"mutex_wait_us":47,"thread_start_us":877,"threads_started":1}
I20260924 00:58:11.835386 20115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1258d0920594663bb850478418ddc63" candidate_uuid: "848888dffcd84201b00f8077992b6c99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522055383d734abf9e6190e1f05156cc" is_pre_election: true
I20260924 00:58:11.835948 20115 raft_consensus.cc:2454] T e1258d0920594663bb850478418ddc63 P 522055383d734abf9e6190e1f05156cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 848888dffcd84201b00f8077992b6c99 in term 0.
I20260924 00:58:11.835968 19972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1258d0920594663bb850478418ddc63" candidate_uuid: "848888dffcd84201b00f8077992b6c99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "063654ebbb284a1abef76e0bef1aef23" is_pre_election: true
I20260924 00:58:11.836537 19972 raft_consensus.cc:2454] T e1258d0920594663bb850478418ddc63 P 063654ebbb284a1abef76e0bef1aef23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 848888dffcd84201b00f8077992b6c99 in term 0.
I20260924 00:58:11.836648 20186 leader_election.cc:302] T e1258d0920594663bb850478418ddc63 P 848888dffcd84201b00f8077992b6c99 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522055383d734abf9e6190e1f05156cc, 848888dffcd84201b00f8077992b6c99; no voters:
I20260924 00:58:11.837139 20322 raft_consensus.cc:2781] T e1258d0920594663bb850478418ddc63 P 848888dffcd84201b00f8077992b6c99 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:58:11.837285 20322 raft_consensus.cc:482] T e1258d0920594663bb850478418ddc63 P 848888dffcd84201b00f8077992b6c99 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:58:11.837389 20322 raft_consensus.cc:3037] T e1258d0920594663bb850478418ddc63 P 848888dffcd84201b00f8077992b6c99 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:58:11.840590 20322 raft_consensus.cc:504] T e1258d0920594663bb850478418ddc63 P 848888dffcd84201b00f8077992b6c99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522055383d734abf9e6190e1f05156cc" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 43303 } } peers { permanent_uuid: "063654ebbb284a1abef76e0bef1aef23" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 39865 } } peers { permanent_uuid: "848888dffcd84201b00f8077992b6c99" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36607 } }
I20260924 00:58:11.841543 20115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1258d0920594663bb850478418ddc63" candidate_uuid: "848888dffcd84201b00f8077992b6c99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522055383d734abf9e6190e1f05156cc"
I20260924 00:58:11.841686 20322 leader_election.cc:288] T e1258d0920594663bb850478418ddc63 P 848888dffcd84201b00f8077992b6c99 [CANDIDATE]: Term 1 election: Requested vote from peers 522055383d734abf9e6190e1f05156cc (127.8.64.130:43303), 063654ebbb284a1abef76e0bef1aef23 (127.8.64.129:39865)
I20260924 00:58:11.841753 20115 raft_consensus.cc:3037] T e1258d0920594663bb850478418ddc63 P 522055383d734abf9e6190e1f05156cc [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:58:11.841753 19972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1258d0920594663bb850478418ddc63" candidate_uuid: "848888dffcd84201b00f8077992b6c99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "063654ebbb284a1abef76e0bef1aef23"
I20260924 00:58:11.842010 19972 raft_consensus.cc:3037] T e1258d0920594663bb850478418ddc63 P 063654ebbb284a1abef76e0bef1aef23 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:58:11.844082 20115 raft_consensus.cc:2454] T e1258d0920594663bb850478418ddc63 P 522055383d734abf9e6190e1f05156cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 848888dffcd84201b00f8077992b6c99 in term 1.
I20260924 00:58:11.844491 20186 leader_election.cc:302] T e1258d0920594663bb850478418ddc63 P 848888dffcd84201b00f8077992b6c99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522055383d734abf9e6190e1f05156cc, 848888dffcd84201b00f8077992b6c99; no voters:
I20260924 00:58:11.844787 20322 raft_consensus.cc:2781] T e1258d0920594663bb850478418ddc63 P 848888dffcd84201b00f8077992b6c99 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:58:11.846019 19972 raft_consensus.cc:2454] T e1258d0920594663bb850478418ddc63 P 063654ebbb284a1abef76e0bef1aef23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 848888dffcd84201b00f8077992b6c99 in term 1.
I20260924 00:58:11.846863 20322 raft_consensus.cc:686] T e1258d0920594663bb850478418ddc63 P 848888dffcd84201b00f8077992b6c99 [term 1 LEADER]: Becoming Leader. State: Replica: 848888dffcd84201b00f8077992b6c99, State: Running, Role: LEADER
I20260924 00:58:11.847450 20322 consensus_queue.cc:237] T e1258d0920594663bb850478418ddc63 P 848888dffcd84201b00f8077992b6c99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522055383d734abf9e6190e1f05156cc" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 43303 } } peers { permanent_uuid: "063654ebbb284a1abef76e0bef1aef23" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 39865 } } peers { permanent_uuid: "848888dffcd84201b00f8077992b6c99" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36607 } }
I20260924 00:58:11.848598 20310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:58:11.834524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:37721 (local address 127.8.64.130:43303)
0924 00:58:11.834714 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0924 00:58:11.834722 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:58:11.835293 (+ 571us) server_negotiation.cc:407] Connection header received
0924 00:58:11.837104 (+ 1811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:58:11.837112 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:58:11.837221 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:58:11.837296 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:58:11.838585 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:58:11.839625 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:58:11.846227 (+ 6602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:58:11.847398 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:58:11.847552 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:58:11.847627 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0924 00:58:11.848053 (+ 426us) server_negotiation.cc:299] Negotiation successful
0924 00:58:11.848176 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260924 00:58:11.849013 20325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:58:11.834485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.130:43303 (local address 127.8.64.129:37721)
0924 00:58:11.835165 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0924 00:58:11.835192 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0924 00:58:11.835402 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:58:11.837532 (+ 2130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:58:11.837539 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:58:11.837573 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:58:11.838002 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:58:11.838016 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:58:11.839765 (+ 1749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:58:11.839774 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:58:11.846036 (+ 6262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:58:11.846053 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:58:11.847708 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:58:11.847714 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:58:11.847875 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:58:11.847915 (+ 40us) client_negotiation.cc:769] Sending connection context
0924 00:58:11.848017 (+ 102us) client_negotiation.cc:240] Negotiation successful
0924 00:58:11.848167 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":407,"threads_started":1}
I20260924 00:58:11.849856 20115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1258d0920594663bb850478418ddc63" candidate_uuid: "063654ebbb284a1abef76e0bef1aef23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522055383d734abf9e6190e1f05156cc" is_pre_election: true
I20260924 00:58:11.850244 20115 raft_consensus.cc:2379] T e1258d0920594663bb850478418ddc63 P 522055383d734abf9e6190e1f05156cc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 063654ebbb284a1abef76e0bef1aef23 in current term 1: Already voted for candidate 848888dffcd84201b00f8077992b6c99 in this term.
I20260924 00:58:11.853403 20326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:58:11.835277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.131:36607 (local address 127.8.64.129:36565)
0924 00:58:11.845354 (+ 10077us) negotiation.cc:107] Waiting for socket to connect
0924 00:58:11.845380 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0924 00:58:11.845596 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:58:11.846223 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:58:11.846230 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:58:11.846262 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:58:11.846835 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:58:11.846849 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:58:11.849095 (+ 2246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:58:11.849101 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:58:11.851149 (+ 2048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:58:11.851162 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:58:11.852679 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:58:11.852685 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:58:11.852854 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:58:11.852886 (+ 32us) client_negotiation.cc:769] Sending connection context
0924 00:58:11.852977 (+ 91us) client_negotiation.cc:240] Negotiation successful
0924 00:58:11.853105 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9881,"thread_start_us":9606,"threads_started":1}
I20260924 00:58:11.853421 20312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:58:11.835648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:36565 (local address 127.8.64.131:36607)
0924 00:58:11.835810 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0924 00:58:11.835815 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:58:11.845478 (+ 9663us) server_negotiation.cc:407] Connection header received
0924 00:58:11.845779 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:58:11.845788 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:58:11.845896 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:58:11.845970 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:58:11.847352 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:58:11.848949 (+ 1597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:58:11.851294 (+ 2345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:58:11.852451 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:58:11.852615 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:58:11.852683 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 00:58:11.853040 (+ 357us) server_negotiation.cc:299] Negotiation successful
0924 00:58:11.853160 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":54}
I20260924 00:58:11.853482 19802 catalog_manager.cc:5495] T e1258d0920594663bb850478418ddc63 P 848888dffcd84201b00f8077992b6c99 reported cstate change: term changed from 0 to 1, leader changed from <none> to 848888dffcd84201b00f8077992b6c99 (127.8.64.131). New cstate: current_term: 1 leader_uuid: "848888dffcd84201b00f8077992b6c99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522055383d734abf9e6190e1f05156cc" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 43303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "063654ebbb284a1abef76e0bef1aef23" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 39865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "848888dffcd84201b00f8077992b6c99" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 36607 } health_report { overall_health: HEALTHY } } }
I20260924 00:58:11.854449 20250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1258d0920594663bb850478418ddc63" candidate_uuid: "063654ebbb284a1abef76e0bef1aef23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "848888dffcd84201b00f8077992b6c99" is_pre_election: true
I20260924 00:58:11.855321 19907 leader_election.cc:302] T e1258d0920594663bb850478418ddc63 P 063654ebbb284a1abef76e0bef1aef23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 063654ebbb284a1abef76e0bef1aef23; no voters: 522055383d734abf9e6190e1f05156cc, 848888dffcd84201b00f8077992b6c99
I20260924 00:58:11.855723 20320 raft_consensus.cc:2726] T e1258d0920594663bb850478418ddc63 P 063654ebbb284a1abef76e0bef1aef23 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 00:58:11.955590 20312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:58:11.948960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52024 (local address 127.8.64.131:36607)
0924 00:58:11.949212 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0924 00:58:11.949219 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:58:11.949438 (+ 219us) server_negotiation.cc:407] Connection header received
0924 00:58:11.949754 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:58:11.949758 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:58:11.949840 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:58:11.950015 (+ 175us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:58:11.950543 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:58:11.951373 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:58:11.953231 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:58:11.953682 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:58:11.953829 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:58:11.954436 (+ 607us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:58:11.954955 (+ 519us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:58:11.955133 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0924 00:58:11.955194 (+ 61us) server_negotiation.cc:299] Negotiation successful
0924 00:58:11.955291 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260924 00:58:11.963788 20115 raft_consensus.cc:1261] T e1258d0920594663bb850478418ddc63 P 522055383d734abf9e6190e1f05156cc [term 1 FOLLOWER]: Refusing update from remote peer 848888dffcd84201b00f8077992b6c99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:58:11.963838 19972 raft_consensus.cc:1261] T e1258d0920594663bb850478418ddc63 P 063654ebbb284a1abef76e0bef1aef23 [term 1 FOLLOWER]: Refusing update from remote peer 848888dffcd84201b00f8077992b6c99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:58:11.964740 20327 consensus_queue.cc:1035] T e1258d0920594663bb850478418ddc63 P 848888dffcd84201b00f8077992b6c99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "063654ebbb284a1abef76e0bef1aef23" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 39865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:58:11.965086 20322 consensus_queue.cc:1035] T e1258d0920594663bb850478418ddc63 P 848888dffcd84201b00f8077992b6c99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522055383d734abf9e6190e1f05156cc" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 43303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:58:11.976502 20331 mvcc.cc:205] Tried to move back new op lower bound from 7332706271074037760 to 7332706270614134784. Current Snapshot: MvccSnapshot[applied={T|T < 7332706271074037760}]
I20260924 00:58:12.004043 19802 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45868:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:58:12.004334 19802 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45868:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:58:12.008507 19802 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 78415e9fdfce4775b378a81b340a5bb1: Sending DeleteTablet for 3 replicas of tablet e1258d0920594663bb850478418ddc63
I20260924 00:58:12.009603 19952 tablet_service.cc:1514] Processing DeleteTablet for tablet e1258d0920594663bb850478418ddc63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:58:12 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:50636
I20260924 00:58:12.009609 20095 tablet_service.cc:1514] Processing DeleteTablet for tablet e1258d0920594663bb850478418ddc63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:58:12 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:59928
I20260924 00:58:12.010030 19951 tablet_service.cc:1514] Processing DeleteTablet for tablet e1258d0920594663bb850478418ddc63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:58:12 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:50636
I20260924 00:58:12.011143 20230 tablet_service.cc:1514] Processing DeleteTablet for tablet e1258d0920594663bb850478418ddc63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:58:12 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:52020
I20260924 00:58:12.012992 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 19883
I20260924 00:58:12.013216 20342 tablet_replica.cc:330] T e1258d0920594663bb850478418ddc63 P 848888dffcd84201b00f8077992b6c99: stopping tablet replica
I20260924 00:58:12.013679 20342 raft_consensus.cc:2229] T e1258d0920594663bb850478418ddc63 P 848888dffcd84201b00f8077992b6c99 [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:58:12.014415 20342 raft_consensus.cc:2258] T e1258d0920594663bb850478418ddc63 P 848888dffcd84201b00f8077992b6c99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:58:12.015373 20341 tablet_replica.cc:330] T e1258d0920594663bb850478418ddc63 P 063654ebbb284a1abef76e0bef1aef23: stopping tablet replica
I20260924 00:58:12.015862 20341 raft_consensus.cc:2229] T e1258d0920594663bb850478418ddc63 P 063654ebbb284a1abef76e0bef1aef23 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:58:12.016458 20341 raft_consensus.cc:2258] T e1258d0920594663bb850478418ddc63 P 063654ebbb284a1abef76e0bef1aef23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:58:12.018496 20342 ts_tablet_manager.cc:1905] T e1258d0920594663bb850478418ddc63 P 848888dffcd84201b00f8077992b6c99: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:58:12.021641 20341 ts_tablet_manager.cc:1905] T e1258d0920594663bb850478418ddc63 P 063654ebbb284a1abef76e0bef1aef23: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:58:12.030786 20340 tablet_replica.cc:330] T e1258d0920594663bb850478418ddc63 P 522055383d734abf9e6190e1f05156cc: stopping tablet replica
I20260924 00:58:12.031251 20340 raft_consensus.cc:2229] T e1258d0920594663bb850478418ddc63 P 522055383d734abf9e6190e1f05156cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:58:12.031739 20340 raft_consensus.cc:2258] T e1258d0920594663bb850478418ddc63 P 522055383d734abf9e6190e1f05156cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:58:12.035776 20340 ts_tablet_manager.cc:1905] T e1258d0920594663bb850478418ddc63 P 522055383d734abf9e6190e1f05156cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:58:12.036871 20341 ts_tablet_manager.cc:1918] T e1258d0920594663bb850478418ddc63 P 063654ebbb284a1abef76e0bef1aef23: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:58:12.037110 20341 log.cc:1196] T e1258d0920594663bb850478418ddc63 P 063654ebbb284a1abef76e0bef1aef23: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-0/wal/wals/e1258d0920594663bb850478418ddc63
I20260924 00:58:12.037755 20341 ts_tablet_manager.cc:1939] T e1258d0920594663bb850478418ddc63 P 063654ebbb284a1abef76e0bef1aef23: Deleting consensus metadata
Sep 24 00:58:12 dist-test-slave-mmhs krb5kdc[19325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211487, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.129@KRBTEST.COM
I20260924 00:58:12.040273 19788 catalog_manager.cc:4841] TS 063654ebbb284a1abef76e0bef1aef23 (127.8.64.129:39865): tablet e1258d0920594663bb850478418ddc63 (table test-table [id=c8496279cdac414facd168dc91b73284]) successfully deleted
W20260924 00:58:12.041510 19788 catalog_manager.cc:4808] TS 063654ebbb284a1abef76e0bef1aef23 (127.8.64.129:39865): delete failed for tablet e1258d0920594663bb850478418ddc63 because the tablet was not found. No further retry: Not found: Tablet not found: e1258d0920594663bb850478418ddc63
I20260924 00:58:12.043287 20340 ts_tablet_manager.cc:1918] T e1258d0920594663bb850478418ddc63 P 522055383d734abf9e6190e1f05156cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:58:12.043449 20340 log.cc:1196] T e1258d0920594663bb850478418ddc63 P 522055383d734abf9e6190e1f05156cc: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-1/wal/wals/e1258d0920594663bb850478418ddc63
I20260924 00:58:12.043946 20340 ts_tablet_manager.cc:1939] T e1258d0920594663bb850478418ddc63 P 522055383d734abf9e6190e1f05156cc: Deleting consensus metadata
I20260924 00:58:12.045774 19790 catalog_manager.cc:4841] TS 522055383d734abf9e6190e1f05156cc (127.8.64.130:43303): tablet e1258d0920594663bb850478418ddc63 (table test-table [id=c8496279cdac414facd168dc91b73284]) successfully deleted
I20260924 00:58:12.046626 20342 ts_tablet_manager.cc:1918] T e1258d0920594663bb850478418ddc63 P 848888dffcd84201b00f8077992b6c99: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:58:12.046909 20311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:58:12.015679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50642 (local address 127.8.64.129:39865)
0924 00:58:12.015880 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0924 00:58:12.015886 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:58:12.027061 (+ 11175us) server_negotiation.cc:407] Connection header received
0924 00:58:12.027409 (+ 348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:58:12.027419 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:58:12.027545 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:58:12.027637 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:58:12.030056 (+ 2419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:58:12.031134 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:58:12.032749 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:58:12.033403 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:58:12.033529 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:58:12.039781 (+ 6252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:58:12.039793 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:58:12.039812 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:58:12.039857 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:58:12.043743 (+ 3886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:58:12.044548 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:58:12.044555 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:58:12.044558 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:58:12.044685 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:58:12.045070 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:58:12.045076 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:58:12.045079 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:58:12.045446 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:58:12.045760 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0924 00:58:12.046326 (+ 566us) server_negotiation.cc:299] Negotiation successful
0924 00:58:12.046545 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260924 00:58:12.047030 20342 log.cc:1196] T e1258d0920594663bb850478418ddc63 P 848888dffcd84201b00f8077992b6c99: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790211343099518-8450-0/minicluster-data/ts-2/wal/wals/e1258d0920594663bb850478418ddc63
I20260924 00:58:12.047614 20342 ts_tablet_manager.cc:1939] T e1258d0920594663bb850478418ddc63 P 848888dffcd84201b00f8077992b6c99: Deleting consensus metadata
I20260924 00:58:12.047694 20012 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:58:12.049311 19790 catalog_manager.cc:4841] TS 848888dffcd84201b00f8077992b6c99 (127.8.64.131:36607): tablet e1258d0920594663bb850478418ddc63 (table test-table [id=c8496279cdac414facd168dc91b73284]) successfully deleted
I20260924 00:58:12.141846 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 19883
I20260924 00:58:12.161685 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 20023
Sep 24 00:58:12 dist-test-slave-mmhs krb5kdc[19325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211487, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.130@KRBTEST.COM
I20260924 00:58:12.177176 20310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:58:12.162513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59944 (local address 127.8.64.130:43303)
0924 00:58:12.162679 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0924 00:58:12.162687 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:58:12.162709 (+ 22us) server_negotiation.cc:407] Connection header received
0924 00:58:12.162928 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:58:12.162934 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:58:12.163009 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:58:12.163140 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:58:12.164357 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:58:12.165078 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:58:12.166396 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:58:12.166817 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:58:12.166947 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:58:12.171189 (+ 4242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:58:12.171204 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:58:12.171220 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:58:12.171254 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:58:12.174403 (+ 3149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:58:12.175300 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:58:12.175307 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:58:12.175309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:58:12.175427 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:58:12.175834 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:58:12.175837 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:58:12.175839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:58:12.176151 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:58:12.176374 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0924 00:58:12.176703 (+ 329us) server_negotiation.cc:299] Negotiation successful
0924 00:58:12.176867 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260924 00:58:12.177870 20156 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:58:12.267266 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 20023
I20260924 00:58:12.285187 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 20167
Sep 24 00:58:12 dist-test-slave-mmhs krb5kdc[19325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211487, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.131@KRBTEST.COM
I20260924 00:58:12.305858 20312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:58:12.286222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52040 (local address 127.8.64.131:36607)
0924 00:58:12.286497 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0924 00:58:12.286504 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:58:12.286524 (+ 20us) server_negotiation.cc:407] Connection header received
0924 00:58:12.286591 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:58:12.286598 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:58:12.286690 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:58:12.286816 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:58:12.288536 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:58:12.289720 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:58:12.291670 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:58:12.292457 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:58:12.292613 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:58:12.298249 (+ 5636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:58:12.298262 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:58:12.298281 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:58:12.298349 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:58:12.302639 (+ 4290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:58:12.303541 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:58:12.303551 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:58:12.303555 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:58:12.303721 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:58:12.304189 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:58:12.304194 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:58:12.304197 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:58:12.304684 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:58:12.305019 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0924 00:58:12.305295 (+ 276us) server_negotiation.cc:299] Negotiation successful
0924 00:58:12.305515 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140}
I20260924 00:58:12.306802 20290 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:58:12.428036 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 20167
I20260924 00:58:12.448055 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 19771
I20260924 00:58:12.449412 19833 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:58:12.560825 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 19771
I20260924 00:58:12.585647 8450 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260924 00:58:12.921172 8450 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260924 00:58:12.921418 8450 mini_ranger.cc:66] Stopping Ranger...
I20260924 00:58:13.266629 8450 mini_ranger.cc:68] Stopped Ranger
2026-09-24 00:58:13.266 UTC [19355] LOG: received smart shutdown request
2026-09-24 00:58:13.270 UTC [19355] LOG: background worker "logical replication launcher" (PID 19364) exited with exit code 1
2026-09-24 00:58:13.271 UTC [19359] LOG: shutting down
2026-09-24 00:58:13.445 UTC [19355] LOG: database system is shut down
2026-09-24T00:58:13Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50170 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:58:13 dist-test-slave-mmhs krb5kdc[20356](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:58:13 dist-test-slave-mmhs krb5kdc[20356](info): set up 2 sockets
Sep 24 00:58:13 dist-test-slave-mmhs krb5kdc[20356](info): commencing operation
krb5kdc: starting...
W20260924 00:58:14.676267 8450 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Sep 24 00:58:14 dist-test-slave-mmhs krb5kdc[20356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211494, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:58:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:58:14Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.8.64.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.8.64.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.8.64.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/rangeradmin_127.8.64.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.8.64.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/rangeradmin_127.8.64.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.8.64.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.8.64.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.8.64.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/rangerlookup_127.8.64.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.8.64.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/rangerlookup_127.8.64.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.64.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.8.64.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.64.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/HTTP_127.8.64.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.64.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/HTTP_127.8.64.148@KRBTEST.COM.keytab.
I20260924 00:58:15.017642 8450 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskDpZYBj/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/postgres -l logfile start
2026-09-24 00:58:17.630 UTC [20386] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-24 00:58:17.630 UTC [20386] LOG: listening on IPv4 address "127.8.64.148", port 52855
2026-09-24 00:58:17.634 UTC [20386] LOG: listening on Unix socket "/tmp/.s.PGSQL.52855"
2026-09-24 00:58:17.655 UTC [20389] LOG: database system was shut down at 2026-09-24 00:58:16 UTC
2026-09-24 00:58:17.659 UTC [20386] LOG: database system is ready to accept connections
I20260924 00:58:18.640148 8450 mini_postgres.cc:96] Postgres bound to 52855
2026-09-24 00:58:18.662 UTC [20397] FATAL: database "slave" does not exist
127.8.64.148:52855 - accepting connections
I20260924 00:58:18.662617 8450 mini_ranger.cc:161] Starting Ranger...
I20260924 00:58:18.688083 8450 mini_ranger.cc:84] Created miniranger Postgres user
I20260924 00:58:19.220095 8450 mini_ranger.cc:87] Created ranger Postgres database
I20260924 00:58:19.220255 8450 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ranger-admin
2026-09-24 00:58:19,516 [I] DB FLAVOR :POSTGRES
2026-09-24 00:58:19,516 [I] --------- Verifying Ranger DB connection ---------
2026-09-24 00:58:19,517 [I] Checking connection..
2026-09-24 00:58:19,517 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.64.148:52855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-24 00:58:19,781 [I] Checking connection passed.
2026-09-24 00:58:19,781 [I] --------- Verifying version history table ---------
2026-09-24 00:58:19,782 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.64.148:52855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 00:58:20,013 [I] Table x_db_version_h does not exist in database ranger
2026-09-24 00:58:20,014 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.64.148:52855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 00:58:20,270 [I] Table x_db_version_h does not exist in database ranger
2026-09-24 00:58:20,270 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-24 00:58:20,271 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.64.148:52855/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-24 00:58:20.494 UTC [20461] WARNING: there is no transaction in progress
2026-09-24 00:58:20,501 [I] create_dbversion_catalog.sql file imported successfully
2026-09-24 00:58:20,501 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.64.148:52855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 00:58:20,745 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-24 00:58:20,745 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-24 00:58:20,745 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.64.148:52855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-24 00:58:21,002 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.64.148:52855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-24 00:58:21,257 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.64.148:52855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-mmhs.c.gcp-upstream.internal','N') ;"
2026-09-24 00:58:21,509 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.64.148:52855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-24 00:58:21,758 [I] Table x_portal_user does not exist in database ranger
2026-09-24 00:58:21,759 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.64.148:52855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-24 00:58:22,000 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-24 00:58:22,000 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.64.148:52855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-24 00:58:22,224 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.64.148:52855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-24 00:58:22,482 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-24 00:58:22,482 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.64.148:52855/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-24 00:58:23.501 UTC [20570] WARNING: there is no transaction in progress
2026-09-24 00:58:23.518 UTC [20570] WARNING: there is no transaction in progress
2026-09-24 00:58:23.535 UTC [20570] WARNING: there is no transaction in progress
2026-09-24 00:58:23.551 UTC [20570] WARNING: there is no transaction in progress
2026-09-24 00:58:23.570 UTC [20570] WARNING: there is no transaction in progress
2026-09-24 00:58:23.590 UTC [20570] WARNING: there is no transaction in progress
2026-09-24 00:58:23.646 UTC [20570] WARNING: there is no transaction in progress
2026-09-24 00:58:23.653 UTC [20570] WARNING: there is no transaction in progress
2026-09-24 00:58:23.668 UTC [20570] WARNING: there is no transaction in progress
2026-09-24 00:58:23.679 UTC [20570] WARNING: there is no transaction in progress
2026-09-24 00:58:23.690 UTC [20570] WARNING: there is no transaction in progress
2026-09-24 00:58:23.703 UTC [20570] WARNING: there is no transaction in progress
2026-09-24 00:58:23.713 UTC [20570] WARNING: there is no transaction in progress
2026-09-24 00:58:23.720 UTC [20570] WARNING: there is no transaction in progress
2026-09-24 00:58:23.727 UTC [20570] WARNING: there is no transaction in progress
2026-09-24 00:58:23.735 UTC [20570] WARNING: there is no transaction in progress
2026-09-24 00:58:24,440 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-24 00:58:24,440 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.64.148:52855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-24 00:58:24,686 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-24 00:58:24,687 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.64.148:52855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-24 00:58:24,931 [I] Table x_portal_user already exists in database 'ranger'
2026-09-24 00:58:24,932 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.64.148:52855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-24 00:58:25,181 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-24 00:58:25,182 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.64.148:52855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-24 00:58:25,438 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.64.148:52855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-24 00:58:25,716 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.64.148:52855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-mmhs.c.gcp-upstream.internal';"
2026-09-24 00:58:25,971 [I] CORE_DB_SCHEMA import status has been updated
2026-09-24 00:58:25,971 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.64.148:52855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-24 00:58:26,203 [I] DB_PATCHES have already been applied
I20260924 00:58:26.209721 8450 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ranger-admin:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskDpZYBj/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260924 00:58:26.209812 8450 mini_ranger.cc:193] Using host: 127.8.64.148
I20260924 00:58:26.225622 8450 mini_ranger.cc:227] Ranger admin URL: http://127.8.64.148:47055
Sep 24, 2026 12:58:26 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 24, 2026 12:58:26 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 24, 2026 12:58:26 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ranger-admin/ews/webapp
Sep 24, 2026 12:58:26 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 24, 2026 12:58:26 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 24, 2026 12:58:27 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ranger-admin/ews/webapp].
Sep 24, 2026 12:58:27 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260924 00:58:27.378187 8450 mini_ranger.cc:160] Time spent starting Ranger: real 8.716s user 0.000s sys 0.043s
Sep 24, 2026 12:58:46 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 24, 2026 12:58:46 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 24, 2026 12:58:46 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 24, 2026 12:58:46 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 24, 2026 12:58:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 24, 2026 12:58:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 24, 2026 12:58:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 24, 2026 12:58:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 24, 2026 12:58:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 24, 2026 12:58:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 24, 2026 12:58:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 24, 2026 12:58:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 24, 2026 12:58:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 24, 2026 12:58:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 24, 2026 12:58:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 24, 2026 12:58:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 24, 2026 12:58:46 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 24, 2026 12:58:47 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260924 00:58:47.632169 8450 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.8.64.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.8.64.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.8.64.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/rangerkms_127.8.64.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.8.64.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/rangerkms_127.8.64.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.64.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.8.64.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.64.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/HTTP_127.8.64.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.64.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/HTTP_127.8.64.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 24 00:58:47 dist-test-slave-mmhs krb5kdc[20356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211527, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260924 00:58:47.807929 8450 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260924 00:58:47.829890 8450 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260924 00:58:48.809875 8450 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260924 00:58:48.809993 8450 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ranger-kms
I20260924 00:58:48.810038 8450 mini_ranger_kms.cc:227] Using postgres at 127.8.64.148:52855
2026-09-24 00:58:49,167 [I] DB FLAVOR :POSTGRES
2026-09-24 00:58:49,167 [I] --------- Verifying Ranger DB connection ---------
2026-09-24 00:58:49,167 [I] Checking connection
2026-09-24 00:58:49,167 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.64.148:52855/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-24 00:58:49,402 [I] connection success
2026-09-24 00:58:49,403 [I] --------- Verifying Ranger DB tables ---------
2026-09-24 00:58:49,403 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-24 00:58:49,403 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.64.148:52855/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-24 00:58:49,678 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-24 00:58:49,678 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-24 00:58:49,678 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-24 00:58:49,678 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.64.148:52855/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-24 00:58:49,937 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260924 00:58:50.371026 8450 mini_ranger_kms.cc:310] Created kms service
I20260924 00:58:50.590508 8450 mini_ranger_kms.cc:326] Created kudu user
I20260924 00:58:50.636621 8450 mini_ranger_kms.cc:343] Created rangerkms user
Sep 24 00:58:50 dist-test-slave-mmhs krb5kdc[20356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211527, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.8.64.148@KRBTEST.COM
I20260924 00:58:51.052426 8450 mini_ranger_kms.cc:384] Added ranger policy
I20260924 00:58:51.052783 8450 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ranger-kms:/tmp/dist-test-taskDpZYBj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskDpZYBj/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260924 00:58:51.052851 8450 mini_ranger_kms.cc:240] Using host: 127.8.64.148
I20260924 00:58:51.072062 8450 mini_ranger_kms.cc:276] Ranger KMS PID: 20751
I20260924 00:58:51.072198 8450 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.8.64.148:47683
Sep 24, 2026 12:58:52 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 24, 2026 12:58:52 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 24, 2026 12:58:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 24, 2026 12:58:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 24, 2026 12:58:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ranger-kms/ews/webapp].
Sep 24, 2026 12:58:52 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260924 00:58:53.396593 8450 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.589s user 0.070s sys 0.304s
I20260924 00:58:53.396891 8450 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260924 00:58:53.397075 8450 mini_ranger_kms.cc:401] 127.8.64.148:47683/kms/v1/keys
Sep 24 00:58:53 dist-test-slave-mmhs krb5kdc[20356](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790211533, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.64.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 24 00:58:53 dist-test-slave-mmhs krb5kdc[20356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211533, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.64.148@KRBTEST.COM for HTTP/127.8.64.148@KRBTEST.COM
Sep 24 00:58:54 dist-test-slave-mmhs krb5kdc[20356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211533, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.64.148@KRBTEST.COM for HTTP/127.8.64.148@KRBTEST.COM
Sep 24 00:58:56 dist-test-slave-mmhs krb5kdc[20356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211536, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.8.64.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
I20260924 00:58:56.631291 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:46357
--webserver_interface=127.8.64.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:34613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:46357
--ranger_config_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.64.148:47683/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:58:56.966086 20803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:58:56.966485 20803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:58:56.966647 20803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:58:56.973915 20803 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:58:56.974071 20803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:58:56.974181 20803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:58:56.974284 20803 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:58:56.974380 20803 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:58:56.984768 20803 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:34613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/master-0/wal
--ranger_kms_url=127.8.64.148:47683/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:46357
--ranger_config_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:46357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:58:56.986114 20803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:58:56.987555 20803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:58:57.001341 20809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:58:57.001456 20808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:58:57.001530 20811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:58:57.002369 20803 server_base.cc:1034] running on GCE node
I20260924 00:58:57.003217 20803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:58:57.004799 20803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:58:57.006039 20803 hybrid_clock.cc:648] HybridClock initialized: now 1790211537005992 us; error 45 us; skew 500 ppm
I20260924 00:58:57.006460 20803 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:58:57 dist-test-slave-mmhs krb5kdc[20356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211537, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:58:57.009877 20803 init.cc:377] Logged in from keytab as kudu/127.8.64.190@KRBTEST.COM (short username kudu)
I20260924 00:58:57.011584 20803 webserver.cc:460] Webserver started at http://127.8.64.190:42803/ using document root <none> and password file <none>
I20260924 00:58:57.012385 20803 fs_manager.cc:359] Metadata directory not provided
I20260924 00:58:57.012526 20803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:58:57.012924 20803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 00:58:57 dist-test-slave-mmhs krb5kdc[20356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211537, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for HTTP/127.8.64.148@KRBTEST.COM
I20260924 00:58:57.064111 20803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "ca0f128d53134d24a09ce3442e33f822"
format_stamp: "Formatted at 2026-09-24 00:58:57 on dist-test-slave-mmhs"
server_key: "0cc639ca718525e04b5c4b3aa438a489"
server_key_iv: "e6228c000850fe4e640cb1c1d3fbedbc"
server_key_version: "kuduclusterkey@0"
I20260924 00:58:57.064988 20803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "ca0f128d53134d24a09ce3442e33f822"
format_stamp: "Formatted at 2026-09-24 00:58:57 on dist-test-slave-mmhs"
server_key: "0cc639ca718525e04b5c4b3aa438a489"
server_key_iv: "e6228c000850fe4e640cb1c1d3fbedbc"
server_key_version: "kuduclusterkey@0"
I20260924 00:58:57.070585 20803 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260924 00:58:57.088848 20818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:58:57.090587 20803 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260924 00:58:57.090803 20803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "ca0f128d53134d24a09ce3442e33f822"
format_stamp: "Formatted at 2026-09-24 00:58:57 on dist-test-slave-mmhs"
server_key: "0cc639ca718525e04b5c4b3aa438a489"
server_key_iv: "e6228c000850fe4e640cb1c1d3fbedbc"
server_key_version: "kuduclusterkey@0"
I20260924 00:58:57.091048 20803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:58:57.112226 20803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:58:57.116531 20803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:58:57.116875 20803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:58:57.136034 20803 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.190:46357
I20260924 00:58:57.136063 20869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.190:46357 every 8 connection(s)
I20260924 00:58:57.137408 20803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 00:58:57.138558 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 20803
I20260924 00:58:57.138808 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 00:58:57.139636 8450 external_mini_cluster.cc:1428] Setting key 26ec13e05baf0fca617661108e128ea3
I20260924 00:58:57.142129 20870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 24 00:58:57 dist-test-slave-mmhs krb5kdc[20356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211536, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:58:57.155409 20870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca0f128d53134d24a09ce3442e33f822: Bootstrap starting.
I20260924 00:58:57.159638 20870 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca0f128d53134d24a09ce3442e33f822: Neither blocks nor log segments found. Creating new log.
I20260924 00:58:57.160941 20870 log.cc:824] T 00000000000000000000000000000000 P ca0f128d53134d24a09ce3442e33f822: Log is configured to *not* fsync() on all Append() calls
I20260924 00:58:57.163529 20873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:58:57.141676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46322 (local address 127.8.64.190:46357)
0924 00:58:57.142492 (+ 816us) server_negotiation.cc:206] Beginning negotiation
0924 00:58:57.142504 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0924 00:58:57.142543 (+ 39us) server_negotiation.cc:407] Connection header received
0924 00:58:57.144085 (+ 1542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:58:57.144105 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:58:57.144469 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:58:57.144803 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:58:57.146534 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:58:57.147891 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:58:57.149700 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:58:57.150392 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:58:57.150517 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:58:57.155024 (+ 4507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:58:57.155036 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:58:57.155058 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:58:57.155101 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:58:57.158910 (+ 3809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:58:57.159622 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:58:57.159633 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:58:57.159640 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:58:57.159838 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:58:57.160202 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:58:57.160207 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:58:57.160210 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:58:57.160899 (+ 689us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:58:57.161189 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0924 00:58:57.161750 (+ 561us) server_negotiation.cc:299] Negotiation successful
0924 00:58:57.162193 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":386,"threads_started":1}
I20260924 00:58:57.164436 20870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca0f128d53134d24a09ce3442e33f822: No bootstrap required, opened a new log
I20260924 00:58:57.172070 20870 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca0f128d53134d24a09ce3442e33f822 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca0f128d53134d24a09ce3442e33f822" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 46357 } }
I20260924 00:58:57.172408 20870 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca0f128d53134d24a09ce3442e33f822 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:58:57.172482 20870 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca0f128d53134d24a09ce3442e33f822 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca0f128d53134d24a09ce3442e33f822, State: Initialized, Role: FOLLOWER
I20260924 00:58:57.173012 20870 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca0f128d53134d24a09ce3442e33f822 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca0f128d53134d24a09ce3442e33f822" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 46357 } }
I20260924 00:58:57.173207 20870 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca0f128d53134d24a09ce3442e33f822 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:58:57.173311 20870 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca0f128d53134d24a09ce3442e33f822 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:58:57.173435 20870 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca0f128d53134d24a09ce3442e33f822 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:58:57.175503 20870 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca0f128d53134d24a09ce3442e33f822 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca0f128d53134d24a09ce3442e33f822" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 46357 } }
I20260924 00:58:57.175943 20870 leader_election.cc:302] T 00000000000000000000000000000000 P ca0f128d53134d24a09ce3442e33f822 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca0f128d53134d24a09ce3442e33f822; no voters:
I20260924 00:58:57.176483 20870 leader_election.cc:288] T 00000000000000000000000000000000 P ca0f128d53134d24a09ce3442e33f822 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:58:57.176623 20875 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca0f128d53134d24a09ce3442e33f822 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:58:57.178540 20875 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca0f128d53134d24a09ce3442e33f822 [term 1 LEADER]: Becoming Leader. State: Replica: ca0f128d53134d24a09ce3442e33f822, State: Running, Role: LEADER
I20260924 00:58:57.178648 20870 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca0f128d53134d24a09ce3442e33f822 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:58:57.179100 20875 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca0f128d53134d24a09ce3442e33f822 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca0f128d53134d24a09ce3442e33f822" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 46357 } }
I20260924 00:58:57.182017 20877 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca0f128d53134d24a09ce3442e33f822 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca0f128d53134d24a09ce3442e33f822. Latest consensus state: current_term: 1 leader_uuid: "ca0f128d53134d24a09ce3442e33f822" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca0f128d53134d24a09ce3442e33f822" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 46357 } } }
I20260924 00:58:57.182080 20876 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca0f128d53134d24a09ce3442e33f822 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca0f128d53134d24a09ce3442e33f822" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca0f128d53134d24a09ce3442e33f822" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 46357 } } }
I20260924 00:58:57.182454 20876 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca0f128d53134d24a09ce3442e33f822 [sys.catalog]: This master's current role is: LEADER
I20260924 00:58:57.182451 20877 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca0f128d53134d24a09ce3442e33f822 [sys.catalog]: This master's current role is: LEADER
I20260924 00:58:57.185822 20885 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:58:57.186224 20870 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260924 00:58:57.202685 20885 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:58:57.213160 20885 catalog_manager.cc:1347] Generated new cluster ID: f6ba4dadd2654bb89b7a4208dabd030d
I20260924 00:58:57.213353 20885 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:58:57.233888 20885 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:58:57.235875 20885 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:58:57.253456 20885 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca0f128d53134d24a09ce3442e33f822: Generated new TSK 0
I20260924 00:58:57.254597 20885 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.64.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab.
Entry for principal kudu/127.8.64.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab.
I20260924 00:58:57.962760 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.64.148:47683/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.129:0
--local_ip_for_outbound_sockets=127.8.64.129
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:46357
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab
--principal=kudu/127.8.64.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:34613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 00:58:58 dist-test-slave-mmhs krb5kdc[20356](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790211538, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260924 00:58:58.426703 20916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:58:58.427191 20916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:58:58.427400 20916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:58:58.440487 20916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:58:58.440649 20916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:58:58.440917 20916 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.129
I20260924 00:58:58.458441 20916 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:34613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.8.64.148:47683/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab
--principal=kudu/127.8.64.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:46357
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.129
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:58:58.460209 20916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:58:58.461985 20916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:58:58.488909 20926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:58:58.489415 20927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:58:58.497684 20916 server_base.cc:1034] running on GCE node
W20260924 00:58:58.497292 20930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:58:58.499010 20916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:58:58.500473 20916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:58:58.501791 20916 hybrid_clock.cc:648] HybridClock initialized: now 1790211538501720 us; error 60 us; skew 500 ppm
I20260924 00:58:58.502195 20916 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:58:58 dist-test-slave-mmhs krb5kdc[20356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211538, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:58:58.506577 20916 init.cc:377] Logged in from keytab as kudu/127.8.64.129@KRBTEST.COM (short username kudu)
I20260924 00:58:58.511886 20916 webserver.cc:460] Webserver started at http://127.8.64.129:37215/ using document root <none> and password file <none>
I20260924 00:58:58.512889 20916 fs_manager.cc:359] Metadata directory not provided
I20260924 00:58:58.513034 20916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:58:58.513322 20916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 00:58:58 dist-test-slave-mmhs krb5kdc[20356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211538, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for HTTP/127.8.64.148@KRBTEST.COM
I20260924 00:58:58.530632 20916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-0/data/instance:
uuid: "6a6bd8bb8ee043268d1826c124777814"
format_stamp: "Formatted at 2026-09-24 00:58:58 on dist-test-slave-mmhs"
server_key: "55397307747786a0289b50b982c133da"
server_key_iv: "c126432cd59664571a3cc6e87d7d052f"
server_key_version: "kuduclusterkey@0"
I20260924 00:58:58.531725 20916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance:
uuid: "6a6bd8bb8ee043268d1826c124777814"
format_stamp: "Formatted at 2026-09-24 00:58:58 on dist-test-slave-mmhs"
server_key: "55397307747786a0289b50b982c133da"
server_key_iv: "c126432cd59664571a3cc6e87d7d052f"
server_key_version: "kuduclusterkey@0"
I20260924 00:58:58.539364 20916 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260924 00:58:58.579013 20936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:58:58.587466 20916 fs_manager.cc:725] Time spent opening block manager: real 0.015s user 0.002s sys 0.004s
I20260924 00:58:58.587760 20916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-0/wal
uuid: "6a6bd8bb8ee043268d1826c124777814"
format_stamp: "Formatted at 2026-09-24 00:58:58 on dist-test-slave-mmhs"
server_key: "55397307747786a0289b50b982c133da"
server_key_iv: "c126432cd59664571a3cc6e87d7d052f"
server_key_version: "kuduclusterkey@0"
I20260924 00:58:58.588088 20916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:58:58.607491 20916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:58:58.613200 20916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:58:58.613659 20916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:58:58.614876 20916 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:58:58.616679 20916 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:58:58.616773 20916 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:58:58.616940 20916 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:58:58.617038 20916 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:58:58.687335 20916 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.129:38481
I20260924 00:58:58.687376 21048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.129:38481 every 8 connection(s)
I20260924 00:58:58.689050 20916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
I20260924 00:58:58.694734 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 20916
I20260924 00:58:58.694945 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance
I20260924 00:58:58.695722 8450 external_mini_cluster.cc:1428] Setting key 7f13592d5e5dac8a02b17a93a8eb19f0
Sep 24 00:58:58 dist-test-slave-mmhs krb5kdc[20356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211538, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.64.130@KRBTEST.COM; defaulting to no policy
I20260924 00:58:58.741142 21051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:58:58.692603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:46357 (local address 127.8.64.129:60189)
0924 00:58:58.708876 (+ 16273us) negotiation.cc:107] Waiting for socket to connect
0924 00:58:58.708933 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0924 00:58:58.712375 (+ 3442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:58:58.715832 (+ 3457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:58:58.715845 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:58:58.716555 (+ 710us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:58:58.717415 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:58:58.717440 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:58:58.722139 (+ 4699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:58:58.722146 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:58:58.724291 (+ 2145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:58:58.724306 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:58:58.725662 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:58:58.725669 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:58:58.725983 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:58:58.726850 (+ 867us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:58:58.726892 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:58:58.731485 (+ 4593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:58:58.736133 (+ 4648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:58:58.736143 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:58:58.736163 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:58:58.736798 (+ 635us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:58:58.737295 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:58:58.737310 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:58:58.737313 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:58:58.737573 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:58:58.738356 (+ 783us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:58:58.738364 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:58:58.738823 (+ 459us) client_negotiation.cc:769] Sending connection context
0924 00:58:58.739153 (+ 330us) client_negotiation.cc:240] Negotiation successful
0924 00:58:58.739721 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15923,"thread_start_us":15662,"threads_started":1}
I20260924 00:58:58.742340 21052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:58:58.693318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:60189 (local address 127.8.64.190:46357)
0924 00:58:58.714048 (+ 20730us) server_negotiation.cc:206] Beginning negotiation
0924 00:58:58.714056 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:58:58.714079 (+ 23us) server_negotiation.cc:407] Connection header received
0924 00:58:58.714145 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:58:58.714153 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:58:58.714255 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:58:58.715032 (+ 777us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:58:58.720781 (+ 5749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:58:58.721943 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:58:58.724514 (+ 2571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:58:58.725195 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:58:58.725324 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:58:58.731854 (+ 6530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:58:58.731860 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:58:58.731864 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:58:58.731919 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:58:58.735916 (+ 3997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:58:58.736963 (+ 1047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:58:58.736969 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:58:58.736973 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:58:58.737127 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:58:58.737714 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:58:58.737719 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:58:58.737723 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:58:58.738094 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:58:58.738238 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0924 00:58:58.741692 (+ 3454us) server_negotiation.cc:299] Negotiation successful
0924 00:58:58.741933 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20586,"thread_start_us":20482,"threads_started":1}
I20260924 00:58:58.744735 21049 heartbeater.cc:347] Connected to a master server at 127.8.64.190:46357
I20260924 00:58:58.745064 21049 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.130@KRBTEST.COM" created.
I20260924 00:58:58.745965 21049 heartbeater.cc:511] Master 127.8.64.190:46357 requested a full tablet report, sending...
I20260924 00:58:58.748723 20835 ts_manager.cc:194] Registered new tserver with Master: 6a6bd8bb8ee043268d1826c124777814 (127.8.64.129:38481)
I20260924 00:58:58.750849 20835 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.129@KRBTEST.COM'} at 127.8.64.129:60189
Sep 24 00:58:58 dist-test-slave-mmhs krb5kdc[20356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211538, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for HTTP/127.8.64.148@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab.
Entry for principal kudu/127.8.64.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab.
I20260924 00:58:58.790179 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.64.148:47683/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.130:0
--local_ip_for_outbound_sockets=127.8.64.130
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:46357
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab
--principal=kudu/127.8.64.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:34613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 00:58:59 dist-test-slave-mmhs krb5kdc[20356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211538, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for HTTP/127.8.64.148@KRBTEST.COM
W20260924 00:58:59.273387 21055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:58:59.273739 21055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:58:59.273895 21055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:58:59.284861 21055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:58:59.285113 21055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:58:59.285362 21055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.130
I20260924 00:58:59.300433 21055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:34613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.8.64.148:47683/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab
--principal=kudu/127.8.64.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:46357
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.130
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:58:59.302275 21055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:58:59.303839 21055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:58:59.339787 21062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:58:59.341743 21063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:58:59.350795 21065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:58:59.351080 21055 server_base.cc:1034] running on GCE node
I20260924 00:58:59.351794 21055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:58:59.352785 21055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:58:59.360285 21055 hybrid_clock.cc:648] HybridClock initialized: now 1790211539360160 us; error 105 us; skew 500 ppm
I20260924 00:58:59.360736 21055 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:58:59 dist-test-slave-mmhs krb5kdc[20356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211539, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:58:59.365096 21055 init.cc:377] Logged in from keytab as kudu/127.8.64.130@KRBTEST.COM (short username kudu)
I20260924 00:58:59.376320 21055 webserver.cc:460] Webserver started at http://127.8.64.130:44383/ using document root <none> and password file <none>
I20260924 00:58:59.377296 21055 fs_manager.cc:359] Metadata directory not provided
I20260924 00:58:59.377509 21055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:58:59.377875 21055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 00:58:59 dist-test-slave-mmhs krb5kdc[20356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211539, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.130@KRBTEST.COM for HTTP/127.8.64.148@KRBTEST.COM
I20260924 00:58:59.402047 21055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-1/data/instance:
uuid: "8a24b18d229349e6bf35c4bd079f41ca"
format_stamp: "Formatted at 2026-09-24 00:58:59 on dist-test-slave-mmhs"
server_key: "d8e7cf9c3d2b85a6bb45767adc529068"
server_key_iv: "57941ea7915f22efc56f8c95f6c8ba56"
server_key_version: "kuduclusterkey@0"
I20260924 00:58:59.403548 21055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance:
uuid: "8a24b18d229349e6bf35c4bd079f41ca"
format_stamp: "Formatted at 2026-09-24 00:58:59 on dist-test-slave-mmhs"
server_key: "d8e7cf9c3d2b85a6bb45767adc529068"
server_key_iv: "57941ea7915f22efc56f8c95f6c8ba56"
server_key_version: "kuduclusterkey@0"
I20260924 00:58:59.422534 21055 fs_manager.cc:691] Time spent creating directory manager: real 0.018s user 0.014s sys 0.005s
I20260924 00:58:59.444443 21079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:58:59.446867 21055 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260924 00:58:59.447100 21055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-1/wal
uuid: "8a24b18d229349e6bf35c4bd079f41ca"
format_stamp: "Formatted at 2026-09-24 00:58:59 on dist-test-slave-mmhs"
server_key: "d8e7cf9c3d2b85a6bb45767adc529068"
server_key_iv: "57941ea7915f22efc56f8c95f6c8ba56"
server_key_version: "kuduclusterkey@0"
I20260924 00:58:59.447381 21055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:58:59.477675 21055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:58:59.482932 21055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:58:59.483276 21055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:58:59.484453 21055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:58:59.486230 21055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:58:59.486357 21055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260924 00:58:59.486479 21055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:58:59.486538 21055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:58:59.525853 21055 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.130:46067
I20260924 00:58:59.525879 21192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.130:46067 every 8 connection(s)
I20260924 00:58:59.527199 21055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
I20260924 00:58:59.537030 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 21055
I20260924 00:58:59.537227 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance
I20260924 00:58:59.537859 8450 external_mini_cluster.cc:1428] Setting key f2cde5b61701af8c916f5c50f678ba42
Sep 24 00:58:59 dist-test-slave-mmhs krb5kdc[20356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211539, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.130@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:58:59.551384 21195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:58:59.529939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:33877 (local address 127.8.64.190:46357)
0924 00:58:59.530436 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0924 00:58:59.530440 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:58:59.531544 (+ 1104us) server_negotiation.cc:407] Connection header received
0924 00:58:59.533777 (+ 2233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:58:59.533786 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:58:59.533899 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:58:59.534012 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:58:59.535661 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:58:59.536425 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:58:59.538319 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:58:59.538895 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:58:59.539041 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:58:59.543337 (+ 4296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:58:59.543346 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:58:59.543354 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:58:59.543404 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:58:59.547406 (+ 4002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:58:59.548382 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:58:59.548390 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:58:59.548397 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:58:59.548556 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:58:59.549084 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:58:59.549091 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:58:59.549096 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:58:59.549501 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:58:59.549669 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0924 00:58:59.550572 (+ 903us) server_negotiation.cc:299] Negotiation successful
0924 00:58:59.550864 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":257,"threads_started":1}
I20260924 00:58:59.552407 21196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:58:59.530613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:46357 (local address 127.8.64.130:33877)
0924 00:58:59.531375 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0924 00:58:59.531413 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0924 00:58:59.533408 (+ 1995us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:58:59.534401 (+ 993us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:58:59.534410 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:58:59.534850 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:58:59.535476 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:58:59.535493 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:58:59.536597 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:58:59.536603 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:58:59.538136 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:58:59.538149 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:58:59.539158 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:58:59.539165 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:58:59.539407 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:58:59.540136 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:58:59.540169 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:58:59.543059 (+ 2890us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:58:59.547599 (+ 4540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:58:59.547609 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:58:59.547629 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:58:59.548225 (+ 596us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:58:59.548676 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:58:59.548681 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:58:59.548685 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:58:59.548938 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:58:59.549646 (+ 708us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:58:59.549654 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:58:59.550098 (+ 444us) client_negotiation.cc:769] Sending connection context
0924 00:58:59.550432 (+ 334us) client_negotiation.cc:240] Negotiation successful
0924 00:58:59.550987 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":315,"threads_started":1}
I20260924 00:58:59.554100 21193 heartbeater.cc:347] Connected to a master server at 127.8.64.190:46357
I20260924 00:58:59.554450 21193 heartbeater.cc:464] Registering TS with master...
I20260924 00:58:59.555311 21193 heartbeater.cc:511] Master 127.8.64.190:46357 requested a full tablet report, sending...
I20260924 00:58:59.557178 20835 ts_manager.cc:194] Registered new tserver with Master: 8a24b18d229349e6bf35c4bd079f41ca (127.8.64.130:46067)
I20260924 00:58:59.558528 20835 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.130@KRBTEST.COM'} at 127.8.64.130:33877
WARNING: no policy specified for kudu/127.8.64.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab.
Entry for principal kudu/127.8.64.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab.
I20260924 00:58:59.598446 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.64.148:47683/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.131:0
--local_ip_for_outbound_sockets=127.8.64.131
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:46357
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab
--principal=kudu/127.8.64.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:34613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 00:58:59.754743 21049 heartbeater.cc:503] Master 127.8.64.190:46357 was elected leader, sending a full tablet report...
W20260924 00:58:59.931259 21199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:58:59.931531 21199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:58:59.931712 21199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:58:59.938956 21199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:58:59.939071 21199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:58:59.939234 21199 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.131
I20260924 00:58:59.948774 21199 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:34613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.8.64.148:47683/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab
--principal=kudu/127.8.64.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:46357
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.131
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:58:59.950006 21199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:58:59.951339 21199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:58:59.963989 21205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:58:59.964040 21207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:58:59.964041 21204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:58:59.964846 21199 server_base.cc:1034] running on GCE node
I20260924 00:58:59.965446 21199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:58:59.966413 21199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:58:59.967630 21199 hybrid_clock.cc:648] HybridClock initialized: now 1790211539967563 us; error 62 us; skew 500 ppm
I20260924 00:58:59.967968 21199 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:58:59 dist-test-slave-mmhs krb5kdc[20356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211539, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:58:59.971047 21199 init.cc:377] Logged in from keytab as kudu/127.8.64.131@KRBTEST.COM (short username kudu)
I20260924 00:58:59.972414 21199 webserver.cc:460] Webserver started at http://127.8.64.131:45201/ using document root <none> and password file <none>
I20260924 00:58:59.973081 21199 fs_manager.cc:359] Metadata directory not provided
I20260924 00:58:59.973213 21199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:58:59.973512 21199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 00:58:59 dist-test-slave-mmhs krb5kdc[20356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211539, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.131@KRBTEST.COM for HTTP/127.8.64.148@KRBTEST.COM
I20260924 00:58:59.990345 21199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-2/data/instance:
uuid: "04eeeca62ebc4a8fa8cdb0e65be2dcc1"
format_stamp: "Formatted at 2026-09-24 00:58:59 on dist-test-slave-mmhs"
server_key: "243f7e83ec4a5aaece5bb90b0b0d39e2"
server_key_iv: "7852f2b799f27d8e253065f38e02e9fe"
server_key_version: "kuduclusterkey@0"
I20260924 00:58:59.991487 21199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance:
uuid: "04eeeca62ebc4a8fa8cdb0e65be2dcc1"
format_stamp: "Formatted at 2026-09-24 00:58:59 on dist-test-slave-mmhs"
server_key: "243f7e83ec4a5aaece5bb90b0b0d39e2"
server_key_iv: "7852f2b799f27d8e253065f38e02e9fe"
server_key_version: "kuduclusterkey@0"
I20260924 00:58:59.997912 21199 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.004s
I20260924 00:59:00.015141 21214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:00.017019 21199 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260924 00:59:00.017208 21199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-2/wal
uuid: "04eeeca62ebc4a8fa8cdb0e65be2dcc1"
format_stamp: "Formatted at 2026-09-24 00:58:59 on dist-test-slave-mmhs"
server_key: "243f7e83ec4a5aaece5bb90b0b0d39e2"
server_key_iv: "7852f2b799f27d8e253065f38e02e9fe"
server_key_version: "kuduclusterkey@0"
I20260924 00:59:00.017459 21199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:59:00.041929 21199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:59:00.046077 21199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:59:00.046351 21199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:59:00.047250 21199 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:59:00.048506 21199 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:59:00.048586 21199 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:00.048681 21199 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:59:00.048722 21199 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:00.082767 21199 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.131:45571
I20260924 00:59:00.082824 21327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.131:45571 every 8 connection(s)
I20260924 00:59:00.084102 21199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
I20260924 00:59:00.093905 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 21199
I20260924 00:59:00.094105 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance
I20260924 00:59:00.094753 8450 external_mini_cluster.cc:1428] Setting key 0e1554a9c6607084e4719321212713c8
Sep 24 00:59:00 dist-test-slave-mmhs krb5kdc[20356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211539, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.131@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:59:00.103974 21330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:00.086570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:43527 (local address 127.8.64.190:46357)
0924 00:59:00.087058 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:00.087062 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:00.087908 (+ 846us) server_negotiation.cc:407] Connection header received
0924 00:59:00.089786 (+ 1878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:00.089793 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:00.089888 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:00.090016 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:00.091405 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:00.092137 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:00.093804 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:00.094435 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:00.094540 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:00.098676 (+ 4136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:00.098681 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:00.098684 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:00.098732 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:00.101207 (+ 2475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:00.101911 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:00.101915 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:00.101918 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:00.102035 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:00.102495 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:00.102498 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:00.102500 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:00.102776 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:00.102885 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:00.103508 (+ 623us) server_negotiation.cc:299] Negotiation successful
0924 00:59:00.103680 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":248,"threads_started":1}
I20260924 00:59:00.104971 21331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:00.087007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:46357 (local address 127.8.64.131:43527)
0924 00:59:00.087745 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:00.087788 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 00:59:00.089519 (+ 1731us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:59:00.090236 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:59:00.090244 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:59:00.090703 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:59:00.091243 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:00.091259 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:00.092295 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:00.092302 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:00.093547 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:00.093560 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:00.094676 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:00.094684 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:00.095016 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:00.095806 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:59:00.095848 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:59:00.098507 (+ 2659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:59:00.101347 (+ 2840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:59:00.101354 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:59:00.101366 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:59:00.101800 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:59:00.102129 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:59:00.102132 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:59:00.102134 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:59:00.102387 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:59:00.102881 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:59:00.102887 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:59:00.103197 (+ 310us) client_negotiation.cc:769] Sending connection context
0924 00:59:00.103422 (+ 225us) client_negotiation.cc:240] Negotiation successful
0924 00:59:00.103794 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":295,"threads_started":1}
I20260924 00:59:00.106535 21328 heartbeater.cc:347] Connected to a master server at 127.8.64.190:46357
I20260924 00:59:00.106809 21328 heartbeater.cc:464] Registering TS with master...
I20260924 00:59:00.107481 21328 heartbeater.cc:511] Master 127.8.64.190:46357 requested a full tablet report, sending...
I20260924 00:59:00.108881 20835 ts_manager.cc:194] Registered new tserver with Master: 04eeeca62ebc4a8fa8cdb0e65be2dcc1 (127.8.64.131:45571)
I20260924 00:59:00.109735 20835 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.131@KRBTEST.COM'} at 127.8.64.131:43527
I20260924 00:59:00.110491 8450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 00:59:00.112108 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 20916
Sep 24 00:59:00 dist-test-slave-mmhs krb5kdc[20356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211536, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.129@KRBTEST.COM
I20260924 00:59:00.129171 21333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:00.113028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58930 (local address 127.8.64.129:38481)
0924 00:59:00.113630 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:00.113637 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:00.113667 (+ 30us) server_negotiation.cc:407] Connection header received
0924 00:59:00.113722 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:00.113728 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:00.113899 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:00.114018 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:00.115481 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:00.116477 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:00.117919 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:00.118824 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:00.118964 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:00.123592 (+ 4628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:00.123603 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:00.123616 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:00.123663 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:00.126624 (+ 2961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:00.127300 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:00.127307 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:00.127309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:00.127440 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:00.127843 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:00.127846 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:00.127848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:00.128141 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:00.128376 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:00.128695 (+ 319us) server_negotiation.cc:299] Negotiation successful
0924 00:59:00.128880 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":311,"threads_started":1}
I20260924 00:59:00.130470 21044 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:59:00.214656 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 20916
I20260924 00:59:00.231489 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 21055
Sep 24 00:59:00 dist-test-slave-mmhs krb5kdc[20356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211536, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.130@KRBTEST.COM
I20260924 00:59:00.247180 21335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:00.232355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44024 (local address 127.8.64.130:46067)
0924 00:59:00.232934 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:00.232941 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:00.232966 (+ 25us) server_negotiation.cc:407] Connection header received
0924 00:59:00.233024 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:00.233030 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:00.233252 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:00.233384 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:00.234654 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:00.235642 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:00.236969 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:00.237501 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:00.237594 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:00.242073 (+ 4479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:00.242083 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:00.242098 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:00.242127 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:00.244707 (+ 2580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:00.245368 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:00.245374 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:00.245376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:00.245502 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:00.245791 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:00.245794 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:00.245796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:00.246101 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:00.246433 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:00.246591 (+ 158us) server_negotiation.cc:299] Negotiation successful
0924 00:59:00.246783 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":296,"threads_started":1}
I20260924 00:59:00.248472 21188 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:59:00.329351 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 21055
I20260924 00:59:00.344956 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 21199
Sep 24 00:59:00 dist-test-slave-mmhs krb5kdc[20356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211536, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.131@KRBTEST.COM
I20260924 00:59:00.361276 21337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:00.345943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55782 (local address 127.8.64.131:45571)
0924 00:59:00.346604 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:00.346610 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:00.346638 (+ 28us) server_negotiation.cc:407] Connection header received
0924 00:59:00.346695 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:00.346701 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:00.346913 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:00.347083 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:00.348393 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:00.349366 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:00.350858 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:00.351427 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:00.351581 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:00.355957 (+ 4376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:00.355969 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:00.355984 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:00.356027 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:00.358718 (+ 2691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:00.359379 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:00.359386 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:00.359389 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:00.359508 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:00.359879 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:00.359883 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:00.359885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:00.360274 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:00.360580 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:00.360757 (+ 177us) server_negotiation.cc:299] Negotiation successful
0924 00:59:00.360960 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":305,"threads_started":1}
I20260924 00:59:00.362843 21323 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:59:00.444875 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 21199
I20260924 00:59:00.460600 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 20803
I20260924 00:59:00.461880 20865 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:59:00.568212 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 20803
I20260924 00:59:00.589637 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:46357
--webserver_interface=127.8.64.190
--webserver_port=42803
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:34613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:46357
--ranger_config_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.64.148:47683/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.64.148:47683/kms with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:59:00.910825 21340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:59:00.911104 21340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:59:00.911190 21340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:59:00.918287 21340 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:59:00.918416 21340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:59:00.918462 21340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:59:00.918519 21340 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:59:00.918568 21340 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:59:00.928313 21340 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:34613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.64.148:47683/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:46357
--ranger_config_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:46357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=42803
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:59:00.929500 21340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:59:00.930795 21340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:59:00.942915 21348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:59:00.942974 21346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:59:00.943007 21345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:59:00.943807 21340 server_base.cc:1034] running on GCE node
I20260924 00:59:00.944541 21340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:59:00.945906 21340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:59:00.947130 21340 hybrid_clock.cc:648] HybridClock initialized: now 1790211540947090 us; error 50 us; skew 500 ppm
I20260924 00:59:00.947477 21340 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:59:00 dist-test-slave-mmhs krb5kdc[20356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211540, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:59:00.953760 21340 init.cc:377] Logged in from keytab as kudu/127.8.64.190@KRBTEST.COM (short username kudu)
I20260924 00:59:00.955250 21340 webserver.cc:460] Webserver started at http://127.8.64.190:42803/ using document root <none> and password file <none>
I20260924 00:59:00.955926 21340 fs_manager.cc:359] Metadata directory not provided
I20260924 00:59:00.956075 21340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 24 00:59:01 dist-test-slave-mmhs krb5kdc[20356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211540, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for HTTP/127.8.64.148@KRBTEST.COM
I20260924 00:59:01.024923 21340 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260924 00:59:01.028110 21356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:01.029486 21340 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260924 00:59:01.029686 21340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "ca0f128d53134d24a09ce3442e33f822"
format_stamp: "Formatted at 2026-09-24 00:58:57 on dist-test-slave-mmhs"
server_key: "0cc639ca718525e04b5c4b3aa438a489"
server_key_iv: "e6228c000850fe4e640cb1c1d3fbedbc"
server_key_version: "kuduclusterkey@0"
I20260924 00:59:01.030685 21340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:59:01.056058 21340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:59:01.059846 21340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:59:01.060171 21340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:59:01.080233 21340 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.190:46357
I20260924 00:59:01.080253 21408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.190:46357 every 8 connection(s)
I20260924 00:59:01.081693 21340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 00:59:01.083678 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 21340
I20260924 00:59:01.084549 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.64.148:47683/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.129:38481
--local_ip_for_outbound_sockets=127.8.64.129
--tserver_master_addrs=127.8.64.190:46357
--webserver_port=37215
--webserver_interface=127.8.64.129
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab
--principal=kudu/127.8.64.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:34613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.64.148:47683/kms with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 00:59:01.088593 21409 sys_catalog.cc:263] Verifying existing consensus state
I20260924 00:59:01.091934 21409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca0f128d53134d24a09ce3442e33f822: Bootstrap starting.
I20260924 00:59:01.110658 21409 log.cc:824] T 00000000000000000000000000000000 P ca0f128d53134d24a09ce3442e33f822: Log is configured to *not* fsync() on all Append() calls
I20260924 00:59:01.119414 21409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca0f128d53134d24a09ce3442e33f822: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 00:59:01.119948 21409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca0f128d53134d24a09ce3442e33f822: Bootstrap complete.
I20260924 00:59:01.129210 21409 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca0f128d53134d24a09ce3442e33f822 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca0f128d53134d24a09ce3442e33f822" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 46357 } }
I20260924 00:59:01.130080 21409 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca0f128d53134d24a09ce3442e33f822 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca0f128d53134d24a09ce3442e33f822, State: Initialized, Role: FOLLOWER
I20260924 00:59:01.130726 21409 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca0f128d53134d24a09ce3442e33f822 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca0f128d53134d24a09ce3442e33f822" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 46357 } }
I20260924 00:59:01.130963 21409 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca0f128d53134d24a09ce3442e33f822 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:59:01.131086 21409 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca0f128d53134d24a09ce3442e33f822 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:59:01.131275 21409 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca0f128d53134d24a09ce3442e33f822 [term 1 FOLLOWER]: Advancing to term 2
I20260924 00:59:01.135246 21409 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca0f128d53134d24a09ce3442e33f822 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca0f128d53134d24a09ce3442e33f822" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 46357 } }
I20260924 00:59:01.135771 21409 leader_election.cc:302] T 00000000000000000000000000000000 P ca0f128d53134d24a09ce3442e33f822 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca0f128d53134d24a09ce3442e33f822; no voters:
I20260924 00:59:01.136413 21409 leader_election.cc:288] T 00000000000000000000000000000000 P ca0f128d53134d24a09ce3442e33f822 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260924 00:59:01.136551 21413 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca0f128d53134d24a09ce3442e33f822 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 00:59:01.137733 21413 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca0f128d53134d24a09ce3442e33f822 [term 2 LEADER]: Becoming Leader. State: Replica: ca0f128d53134d24a09ce3442e33f822, State: Running, Role: LEADER
I20260924 00:59:01.138337 21413 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca0f128d53134d24a09ce3442e33f822 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca0f128d53134d24a09ce3442e33f822" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 46357 } }
I20260924 00:59:01.139797 21413 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca0f128d53134d24a09ce3442e33f822 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca0f128d53134d24a09ce3442e33f822. Latest consensus state: current_term: 2 leader_uuid: "ca0f128d53134d24a09ce3442e33f822" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca0f128d53134d24a09ce3442e33f822" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 46357 } } }
I20260924 00:59:01.140259 21413 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca0f128d53134d24a09ce3442e33f822 [sys.catalog]: This master's current role is: LEADER
I20260924 00:59:01.140530 21415 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca0f128d53134d24a09ce3442e33f822 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ca0f128d53134d24a09ce3442e33f822" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca0f128d53134d24a09ce3442e33f822" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 46357 } } }
I20260924 00:59:01.140866 21415 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca0f128d53134d24a09ce3442e33f822 [sys.catalog]: This master's current role is: LEADER
I20260924 00:59:01.144819 21409 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca0f128d53134d24a09ce3442e33f822 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:59:01.147372 21417 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:59:01.152643 21417 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:59:01.154052 21417 catalog_manager.cc:1259] Loaded cluster ID: f6ba4dadd2654bb89b7a4208dabd030d
I20260924 00:59:01.154232 21417 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:59:01.158277 21417 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:59:01.159890 21417 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ca0f128d53134d24a09ce3442e33f822: Loaded TSK: 0
I20260924 00:59:01.161592 21417 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 00:59:01.167053 21409 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260924 00:59:01.532688 21411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:59:01.533219 21411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:59:01.533420 21411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:59:01.546130 21411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:59:01.546401 21411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:59:01.546656 21411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.129
I20260924 00:59:01.562798 21411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:34613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.64.148:47683/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab
--principal=kudu/127.8.64.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.129:38481
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.64.129
--webserver_port=37215
--tserver_master_addrs=127.8.64.190:46357
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.129
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:59:01.564538 21411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:59:01.566366 21411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:59:01.607359 21446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:59:01.609076 21447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:59:01.613793 21411 server_base.cc:1034] running on GCE node
W20260924 00:59:01.614933 21449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:59:01.615895 21411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:59:01.618695 21411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:59:01.619987 21411 hybrid_clock.cc:648] HybridClock initialized: now 1790211541619826 us; error 149 us; skew 500 ppm
I20260924 00:59:01.620407 21411 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:59:01 dist-test-slave-mmhs krb5kdc[20356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211541, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:59:01.624806 21411 init.cc:377] Logged in from keytab as kudu/127.8.64.129@KRBTEST.COM (short username kudu)
I20260924 00:59:01.632064 21411 webserver.cc:460] Webserver started at http://127.8.64.129:37215/ using document root <none> and password file <none>
I20260924 00:59:01.633198 21411 fs_manager.cc:359] Metadata directory not provided
I20260924 00:59:01.633421 21411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 24 00:59:01 dist-test-slave-mmhs krb5kdc[20356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211541, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for HTTP/127.8.64.148@KRBTEST.COM
I20260924 00:59:01.667565 21411 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.001s
I20260924 00:59:01.670730 21457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:01.672286 21411 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260924 00:59:01.672482 21411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-0/wal
uuid: "6a6bd8bb8ee043268d1826c124777814"
format_stamp: "Formatted at 2026-09-24 00:58:58 on dist-test-slave-mmhs"
server_key: "55397307747786a0289b50b982c133da"
server_key_iv: "c126432cd59664571a3cc6e87d7d052f"
server_key_version: "kuduclusterkey@0"
I20260924 00:59:01.673772 21411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:59:01.736131 21411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:59:01.742004 21411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:59:01.742434 21411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:59:01.748214 21411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:59:01.750190 21411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:59:01.750461 21411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260924 00:59:01.750619 21411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:59:01.750727 21411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:01.819974 21411 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.129:38481
I20260924 00:59:01.820024 21571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.129:38481 every 8 connection(s)
I20260924 00:59:01.821508 21411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
I20260924 00:59:01.822196 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 21411
I20260924 00:59:01.823055 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.64.148:47683/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.130:46067
--local_ip_for_outbound_sockets=127.8.64.130
--tserver_master_addrs=127.8.64.190:46357
--webserver_port=44383
--webserver_interface=127.8.64.130
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab
--principal=kudu/127.8.64.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:34613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.64.148:47683/kms with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 00:59:01 dist-test-slave-mmhs krb5kdc[20356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211541, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:59:01.858902 21574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:01.826014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:46357 (local address 127.8.64.129:40749)
0924 00:59:01.831554 (+ 5540us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:01.831608 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0924 00:59:01.834362 (+ 2754us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:59:01.836860 (+ 2498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:59:01.836871 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:59:01.837495 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:59:01.838341 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:01.838369 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:01.841020 (+ 2651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:01.841026 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:01.843123 (+ 2097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:01.843139 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:01.844827 (+ 1688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:01.844835 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:01.845137 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:01.846019 (+ 882us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:59:01.846052 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:59:01.849138 (+ 3086us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:59:01.853807 (+ 4669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:59:01.853817 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:59:01.853837 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:59:01.854451 (+ 614us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:59:01.854985 (+ 534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:59:01.854990 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:59:01.854994 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:59:01.855237 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:59:01.856235 (+ 998us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:59:01.856244 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:59:01.856630 (+ 386us) client_negotiation.cc:769] Sending connection context
0924 00:59:01.856930 (+ 300us) client_negotiation.cc:240] Negotiation successful
0924 00:59:01.857535 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5252,"thread_start_us":5001,"threads_started":1}
I20260924 00:59:01.860050 21575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:01.827598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:40749 (local address 127.8.64.190:46357)
0924 00:59:01.830951 (+ 3353us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:01.830977 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:01.834747 (+ 3770us) server_negotiation.cc:407] Connection header received
0924 00:59:01.835202 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:01.835224 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:01.835703 (+ 479us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:01.836082 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:01.838603 (+ 2521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:01.840818 (+ 2215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:01.843299 (+ 2481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:01.844545 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:01.844784 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:01.849509 (+ 4725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:01.849521 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:01.849543 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:01.849597 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:01.853625 (+ 4028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:01.854636 (+ 1011us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:01.854645 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:01.854654 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:01.854862 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:01.855378 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:01.855385 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:01.855388 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:01.856080 (+ 692us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:01.856730 (+ 650us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:01.857181 (+ 451us) server_negotiation.cc:299] Negotiation successful
0924 00:59:01.857578 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3031,"thread_start_us":2762,"threads_started":1}
I20260924 00:59:01.863428 21572 heartbeater.cc:347] Connected to a master server at 127.8.64.190:46357
I20260924 00:59:01.863754 21572 heartbeater.cc:464] Registering TS with master...
I20260924 00:59:01.864576 21572 heartbeater.cc:511] Master 127.8.64.190:46357 requested a full tablet report, sending...
W20260924 00:59:01.867920 21572 heartbeater.cc:644] Failed to heartbeat to 127.8.64.190:46357 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260924 00:59:01.868131 21572 heartbeater.cc:464] Registering TS with master...
I20260924 00:59:01.868391 21572 heartbeater.cc:511] Master 127.8.64.190:46357 requested a full tablet report, sending...
I20260924 00:59:01.869719 21572 heartbeater.cc:464] Registering TS with master...
I20260924 00:59:01.869967 21572 heartbeater.cc:511] Master 127.8.64.190:46357 requested a full tablet report, sending...
W20260924 00:59:01.871088 21572 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 00:59:02.226912 21576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:59:02.227303 21576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:59:02.227509 21576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:59:02.237576 21576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:59:02.237679 21576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:59:02.237850 21576 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.130
I20260924 00:59:02.248632 21576 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:34613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.64.148:47683/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab
--principal=kudu/127.8.64.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.130:46067
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.64.130
--webserver_port=44383
--tserver_master_addrs=127.8.64.190:46357
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.130
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:59:02.250202 21576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:59:02.251960 21576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:59:02.284232 21590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:59:02.284258 21592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:59:02.286931 21594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:59:02.288594 21576 server_base.cc:1034] running on GCE node
I20260924 00:59:02.289403 21576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:59:02.310956 21576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:59:02.312314 21576 hybrid_clock.cc:648] HybridClock initialized: now 1790211542312228 us; error 86 us; skew 500 ppm
I20260924 00:59:02.312832 21576 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:59:02 dist-test-slave-mmhs krb5kdc[20356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211542, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:59:02.317395 21576 init.cc:377] Logged in from keytab as kudu/127.8.64.130@KRBTEST.COM (short username kudu)
I20260924 00:59:02.319816 21576 webserver.cc:460] Webserver started at http://127.8.64.130:44383/ using document root <none> and password file <none>
I20260924 00:59:02.320783 21576 fs_manager.cc:359] Metadata directory not provided
I20260924 00:59:02.320935 21576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 24 00:59:02 dist-test-slave-mmhs krb5kdc[20356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211542, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.130@KRBTEST.COM for HTTP/127.8.64.148@KRBTEST.COM
I20260924 00:59:02.352562 21576 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.006s sys 0.000s
I20260924 00:59:02.356369 21603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:02.358256 21576 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260924 00:59:02.358459 21576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-1/wal
uuid: "8a24b18d229349e6bf35c4bd079f41ca"
format_stamp: "Formatted at 2026-09-24 00:58:59 on dist-test-slave-mmhs"
server_key: "d8e7cf9c3d2b85a6bb45767adc529068"
server_key_iv: "57941ea7915f22efc56f8c95f6c8ba56"
server_key_version: "kuduclusterkey@0"
I20260924 00:59:02.359881 21576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Sep 24 00:59:02 dist-test-slave-mmhs krb5kdc[20356](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790211542, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:59:02.395955 21576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:59:02.403203 21576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:59:02.403538 21576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:59:02.410833 21576 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:59:02.412510 21576 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:59:02.412647 21576 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:02.412797 21576 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:59:02.412891 21576 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:02.459676 21576 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.130:46067
I20260924 00:59:02.459775 21716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.130:46067 every 8 connection(s)
I20260924 00:59:02.461560 21576 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
I20260924 00:59:02.471658 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 21576
I20260924 00:59:02.472460 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.64.148:47683/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.131:45571
--local_ip_for_outbound_sockets=127.8.64.131
--tserver_master_addrs=127.8.64.190:46357
--webserver_port=45201
--webserver_interface=127.8.64.131
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab
--principal=kudu/127.8.64.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:34613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.64.148:47683/kms with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 00:59:02 dist-test-slave-mmhs krb5kdc[20356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211542, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.130@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:59:02.526374 21720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:02.465809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:57145 (local address 127.8.64.190:46357)
0924 00:59:02.502277 (+ 36468us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:02.502285 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:02.502336 (+ 51us) server_negotiation.cc:407] Connection header received
0924 00:59:02.504360 (+ 2024us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:02.504371 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:02.504531 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:02.504635 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:02.508652 (+ 4017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:02.510094 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:02.513121 (+ 3027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:02.513853 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:02.513958 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:02.518831 (+ 4873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:02.518838 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:02.518842 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:02.518885 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:02.522513 (+ 3628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:02.523517 (+ 1004us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:02.523524 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:02.523528 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:02.523694 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:02.524247 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:02.524254 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:02.524257 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:02.524640 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:02.524796 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:02.525735 (+ 939us) server_negotiation.cc:299] Negotiation successful
0924 00:59:02.525962 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36314,"thread_start_us":37586,"threads_started":1}
I20260924 00:59:02.527895 21719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:02.465163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:46357 (local address 127.8.64.130:57145)
0924 00:59:02.498763 (+ 33600us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:02.498844 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0924 00:59:02.503937 (+ 5093us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:59:02.506034 (+ 2097us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:59:02.506051 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:59:02.506873 (+ 822us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:59:02.507951 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:02.507987 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:02.510394 (+ 2407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:02.510403 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:02.512930 (+ 2527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:02.512947 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:02.514360 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:02.514368 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:02.514698 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:02.515530 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:59:02.515564 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:59:02.518531 (+ 2967us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:59:02.522707 (+ 4176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:59:02.522717 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:59:02.522738 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:59:02.523358 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:59:02.523821 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:59:02.523826 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:59:02.523830 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:59:02.524102 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:59:02.524788 (+ 686us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:59:02.524799 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:59:02.525246 (+ 447us) client_negotiation.cc:769] Sending connection context
0924 00:59:02.525580 (+ 334us) client_negotiation.cc:240] Negotiation successful
0924 00:59:02.526177 (+ 597us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33130,"thread_start_us":37086,"threads_started":1}
I20260924 00:59:02.530527 21717 heartbeater.cc:347] Connected to a master server at 127.8.64.190:46357
I20260924 00:59:02.530843 21717 heartbeater.cc:464] Registering TS with master...
I20260924 00:59:02.531711 21717 heartbeater.cc:511] Master 127.8.64.190:46357 requested a full tablet report, sending...
I20260924 00:59:02.533852 21373 ts_manager.cc:194] Registered new tserver with Master: 8a24b18d229349e6bf35c4bd079f41ca (127.8.64.130:46067)
I20260924 00:59:02.536207 21373 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.130@KRBTEST.COM'} at 127.8.64.130:57145
Sep 24 00:59:02 dist-test-slave-mmhs krb5kdc[20356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211542, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for HTTP/127.8.64.148@KRBTEST.COM
I20260924 00:59:02.871229 21572 heartbeater.cc:464] Registering TS with master...
I20260924 00:59:02.871665 21572 heartbeater.cc:511] Master 127.8.64.190:46357 requested a full tablet report, sending...
I20260924 00:59:02.873071 21373 ts_manager.cc:194] Registered new tserver with Master: 6a6bd8bb8ee043268d1826c124777814 (127.8.64.129:38481)
I20260924 00:59:02.874280 21373 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.129@KRBTEST.COM'} at 127.8.64.129:40749
W20260924 00:59:02.904853 21721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:59:02.905337 21721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:59:02.905586 21721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:59:02.916855 21721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:59:02.917076 21721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:59:02.917305 21721 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.131
I20260924 00:59:02.932924 21721 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:34613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.64.148:47683/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab
--principal=kudu/127.8.64.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.131:45571
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.64.131
--webserver_port=45201
--tserver_master_addrs=127.8.64.190:46357
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.131
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:59:02.934832 21721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:59:02.936267 21721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:59:02.953332 21727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:59:02.962752 21726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:59:02.963994 21730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:59:02.964564 21721 server_base.cc:1034] running on GCE node
I20260924 00:59:02.965313 21721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:59:02.966549 21721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:59:02.967849 21721 hybrid_clock.cc:648] HybridClock initialized: now 1790211542967784 us; error 47 us; skew 500 ppm
I20260924 00:59:02.968219 21721 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:59:02 dist-test-slave-mmhs krb5kdc[20356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211542, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:59:02.972306 21721 init.cc:377] Logged in from keytab as kudu/127.8.64.131@KRBTEST.COM (short username kudu)
I20260924 00:59:02.979836 21721 webserver.cc:460] Webserver started at http://127.8.64.131:45201/ using document root <none> and password file <none>
I20260924 00:59:02.980767 21721 fs_manager.cc:359] Metadata directory not provided
I20260924 00:59:02.980924 21721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 24 00:59:02 dist-test-slave-mmhs krb5kdc[20356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211542, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.131@KRBTEST.COM for HTTP/127.8.64.148@KRBTEST.COM
I20260924 00:59:03.023729 21721 fs_manager.cc:709] Time spent opening directory manager: real 0.014s user 0.000s sys 0.012s
I20260924 00:59:03.027518 21737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:03.029445 21721 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260924 00:59:03.029630 21721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-2/wal
uuid: "04eeeca62ebc4a8fa8cdb0e65be2dcc1"
format_stamp: "Formatted at 2026-09-24 00:58:59 on dist-test-slave-mmhs"
server_key: "243f7e83ec4a5aaece5bb90b0b0d39e2"
server_key_iv: "7852f2b799f27d8e253065f38e02e9fe"
server_key_version: "kuduclusterkey@0"
I20260924 00:59:03.031028 21721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:59:03.056988 21721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:59:03.062412 21721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:59:03.062726 21721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:59:03.064029 21721 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:59:03.065816 21721 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:59:03.065917 21721 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:03.066018 21721 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:59:03.066089 21721 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:03.134393 21721 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.131:45571
I20260924 00:59:03.134443 21850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.131:45571 every 8 connection(s)
I20260924 00:59:03.136086 21721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
I20260924 00:59:03.141377 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 21721
I20260924 00:59:03.171748 21854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:03.144558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46324 (local address 127.8.64.190:46357)
0924 00:59:03.146837 (+ 2279us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:03.146846 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:03.146875 (+ 29us) server_negotiation.cc:407] Connection header received
0924 00:59:03.146961 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:03.146970 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:03.147111 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:03.149893 (+ 2782us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:03.150127 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:03.151663 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:03.156340 (+ 4677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:03.157008 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:03.157119 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:03.162031 (+ 4912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:03.162037 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:03.162041 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:03.162087 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:03.168655 (+ 6568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:03.169448 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:03.169456 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:03.169463 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:03.169634 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:03.170017 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:03.170023 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:03.170026 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:03.170397 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:03.170663 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:03.170965 (+ 302us) server_negotiation.cc:299] Negotiation successful
0924 00:59:03.171181 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2117,"thread_start_us":277,"threads_started":1}
Sep 24 00:59:03 dist-test-slave-mmhs krb5kdc[20356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211542, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.131@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:59:03.181929 21856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:03.142257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:60207 (local address 127.8.64.190:46357)
0924 00:59:03.145056 (+ 2799us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:03.145063 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:03.152566 (+ 7503us) server_negotiation.cc:407] Connection header received
0924 00:59:03.156902 (+ 4336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:03.156908 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:03.157008 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:03.157096 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:03.163565 (+ 6469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:03.165223 (+ 1658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:03.167872 (+ 2649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:03.168600 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:03.168714 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:03.175140 (+ 6426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:03.175147 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:03.175152 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:03.175192 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:03.178525 (+ 3333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:03.179450 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:03.179456 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:03.179459 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:03.179601 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:03.180131 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:03.180136 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:03.180139 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:03.180409 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:03.180637 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:03.181340 (+ 703us) server_negotiation.cc:299] Negotiation successful
0924 00:59:03.181541 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2685,"thread_start_us":4229,"threads_started":1}
Sep 24 00:59:03 dist-test-slave-mmhs krb5kdc[20356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211542, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for HTTP/127.8.64.148@KRBTEST.COM
I20260924 00:59:03.183899 21853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:03.140279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:46357 (local address 127.8.64.131:60207)
0924 00:59:03.152412 (+ 12133us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:03.152471 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0924 00:59:03.156573 (+ 4102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:59:03.161345 (+ 4772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:59:03.161358 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:59:03.161985 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:59:03.163298 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:03.163327 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:03.165435 (+ 2108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:03.165441 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:03.167680 (+ 2239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:03.167698 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:03.170042 (+ 2344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:03.170053 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:03.170388 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:03.172156 (+ 1768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:59:03.172189 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:59:03.174824 (+ 2635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:59:03.178681 (+ 3857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:59:03.178690 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:59:03.178707 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:59:03.179306 (+ 599us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:59:03.179769 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:59:03.179772 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:59:03.179774 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:59:03.179997 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:59:03.180570 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:59:03.180578 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:59:03.180966 (+ 388us) client_negotiation.cc:769] Sending connection context
0924 00:59:03.181242 (+ 276us) client_negotiation.cc:240] Negotiation successful
0924 00:59:03.181774 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11832,"thread_start_us":11585,"threads_started":1}
I20260924 00:59:03.185286 21851 heartbeater.cc:347] Connected to a master server at 127.8.64.190:46357
I20260924 00:59:03.185613 21851 heartbeater.cc:464] Registering TS with master...
I20260924 00:59:03.186506 21851 heartbeater.cc:511] Master 127.8.64.190:46357 requested a full tablet report, sending...
I20260924 00:59:03.188112 21373 ts_manager.cc:194] Registered new tserver with Master: 04eeeca62ebc4a8fa8cdb0e65be2dcc1 (127.8.64.131:45571)
I20260924 00:59:03.189234 21373 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.131@KRBTEST.COM'} at 127.8.64.131:60207
I20260924 00:59:03.197877 8450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 00:59:03.238831 21856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:03.217195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46326 (local address 127.8.64.190:46357)
0924 00:59:03.217384 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:03.217393 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:03.222889 (+ 5496us) server_negotiation.cc:407] Connection header received
0924 00:59:03.223093 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:03.223099 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:03.223192 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:03.223291 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:03.225269 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:03.226394 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:03.228465 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:03.229028 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:03.229185 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:03.231881 (+ 2696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:03.231889 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:03.231899 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:03.231935 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:03.235458 (+ 3523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:03.236333 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:03.236339 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:03.236343 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:03.236486 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:03.236966 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:03.236971 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:03.236974 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:03.237380 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:03.237608 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:03.238206 (+ 598us) server_negotiation.cc:299] Negotiation successful
0924 00:59:03.238485 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260924 00:59:03.246044 21373 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46326:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:59:03.257768 21373 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:59:03.329722 21873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:03.315787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.131:45571 (local address 127.0.0.1:55786)
0924 00:59:03.320566 (+ 4779us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:03.320594 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0924 00:59:03.320729 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:59:03.321818 (+ 1089us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:59:03.321823 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:59:03.321847 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:59:03.322220 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:03.322230 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:03.324192 (+ 1962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:03.324197 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:03.326190 (+ 1993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:03.326206 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:03.328733 (+ 2527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:03.328739 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:03.328945 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:03.329015 (+ 70us) client_negotiation.cc:769] Sending connection context
0924 00:59:03.329152 (+ 137us) client_negotiation.cc:240] Negotiation successful
0924 00:59:03.329339 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4663,"thread_start_us":5025,"threads_started":1}
I20260924 00:59:03.330801 21872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:03.310091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55786 (local address 127.8.64.131:45571)
0924 00:59:03.317734 (+ 7643us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:03.317743 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:03.320692 (+ 2949us) server_negotiation.cc:407] Connection header received
0924 00:59:03.321071 (+ 379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:03.321080 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:03.321335 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:03.321496 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:59:03.322454 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:03.324024 (+ 1570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:03.326714 (+ 2690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:03.328201 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:03.328294 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:03.328535 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:03.330229 (+ 1694us) server_negotiation.cc:299] Negotiation successful
0924 00:59:03.330435 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7450,"thread_start_us":7308,"threads_started":1}
I20260924 00:59:03.332758 21868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:03.286114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.130:46067 (local address 127.0.0.1:44036)
0924 00:59:03.309619 (+ 23505us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:03.309644 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0924 00:59:03.314744 (+ 5100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:59:03.315850 (+ 1106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:59:03.315857 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:59:03.315886 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:59:03.316347 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:03.316362 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:03.319886 (+ 3524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:03.319894 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:03.322520 (+ 2626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:03.322537 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:03.332005 (+ 9468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:03.332012 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:03.332199 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:03.332229 (+ 30us) client_negotiation.cc:769] Sending connection context
0924 00:59:03.332313 (+ 84us) client_negotiation.cc:240] Negotiation successful
0924 00:59:03.332443 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23379,"thread_start_us":16611,"threads_started":1}
I20260924 00:59:03.333454 21871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:03.287067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44036 (local address 127.8.64.130:46067)
0924 00:59:03.308003 (+ 20936us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:03.308016 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:03.314906 (+ 6890us) server_negotiation.cc:407] Connection header received
0924 00:59:03.315041 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:03.315053 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:03.315399 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:03.315615 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:59:03.316562 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:03.319683 (+ 3121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:03.324838 (+ 5155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:03.326362 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:03.331631 (+ 5269us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:03.331853 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:03.332971 (+ 1118us) server_negotiation.cc:299] Negotiation successful
0924 00:59:03.333174 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20775,"thread_start_us":20680,"threads_started":1}
I20260924 00:59:03.334661 21869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:03.285947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:38481 (local address 127.0.0.1:58934)
0924 00:59:03.302188 (+ 16241us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:03.302247 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0924 00:59:03.302497 (+ 250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:59:03.311249 (+ 8752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:59:03.311262 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:59:03.311311 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:59:03.311840 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:03.311863 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:03.326553 (+ 14690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:03.326559 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:03.328529 (+ 1970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:03.328543 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:03.333849 (+ 5306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:03.333857 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:03.334034 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:03.334066 (+ 32us) client_negotiation.cc:769] Sending connection context
0924 00:59:03.334143 (+ 77us) client_negotiation.cc:240] Negotiation successful
0924 00:59:03.334264 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15890,"thread_start_us":23419,"threads_started":1}
I20260924 00:59:03.337203 21870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:03.286533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58934 (local address 127.8.64.129:38481)
0924 00:59:03.308600 (+ 22067us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:03.308612 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:03.308653 (+ 41us) server_negotiation.cc:407] Connection header received
0924 00:59:03.308794 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:03.308805 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:03.310906 (+ 2101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:03.312138 (+ 1232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:59:03.312463 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:03.324661 (+ 12198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:03.329893 (+ 5232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:03.331404 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:03.334788 (+ 3384us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:03.335018 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:03.335107 (+ 89us) server_negotiation.cc:299] Negotiation successful
0924 00:59:03.335270 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21862,"thread_start_us":21654,"threads_started":1}
I20260924 00:59:03.335990 21785 tablet_service.cc:1467] Processing CreateTablet for tablet 5d83f06ff1a044eb83bf454e274c3612 (DEFAULT_TABLE table=test-table [id=b12f584c38434db5b871849557c570e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:59:03.339164 21785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d83f06ff1a044eb83bf454e274c3612. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:59:03.340466 21651 tablet_service.cc:1467] Processing CreateTablet for tablet 5d83f06ff1a044eb83bf454e274c3612 (DEFAULT_TABLE table=test-table [id=b12f584c38434db5b871849557c570e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:59:03.341825 21651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d83f06ff1a044eb83bf454e274c3612. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:59:03.343941 21506 tablet_service.cc:1467] Processing CreateTablet for tablet 5d83f06ff1a044eb83bf454e274c3612 (DEFAULT_TABLE table=test-table [id=b12f584c38434db5b871849557c570e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:59:03.346773 21506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d83f06ff1a044eb83bf454e274c3612. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:59:03.360733 21874 tablet_bootstrap.cc:492] T 5d83f06ff1a044eb83bf454e274c3612 P 04eeeca62ebc4a8fa8cdb0e65be2dcc1: Bootstrap starting.
I20260924 00:59:03.367703 21874 tablet_bootstrap.cc:654] T 5d83f06ff1a044eb83bf454e274c3612 P 04eeeca62ebc4a8fa8cdb0e65be2dcc1: Neither blocks nor log segments found. Creating new log.
I20260924 00:59:03.369269 21874 log.cc:824] T 5d83f06ff1a044eb83bf454e274c3612 P 04eeeca62ebc4a8fa8cdb0e65be2dcc1: Log is configured to *not* fsync() on all Append() calls
I20260924 00:59:03.370077 21875 tablet_bootstrap.cc:492] T 5d83f06ff1a044eb83bf454e274c3612 P 8a24b18d229349e6bf35c4bd079f41ca: Bootstrap starting.
I20260924 00:59:03.371106 21876 tablet_bootstrap.cc:492] T 5d83f06ff1a044eb83bf454e274c3612 P 6a6bd8bb8ee043268d1826c124777814: Bootstrap starting.
I20260924 00:59:03.376603 21876 tablet_bootstrap.cc:654] T 5d83f06ff1a044eb83bf454e274c3612 P 6a6bd8bb8ee043268d1826c124777814: Neither blocks nor log segments found. Creating new log.
I20260924 00:59:03.378363 21875 tablet_bootstrap.cc:654] T 5d83f06ff1a044eb83bf454e274c3612 P 8a24b18d229349e6bf35c4bd079f41ca: Neither blocks nor log segments found. Creating new log.
I20260924 00:59:03.378283 21876 log.cc:824] T 5d83f06ff1a044eb83bf454e274c3612 P 6a6bd8bb8ee043268d1826c124777814: Log is configured to *not* fsync() on all Append() calls
I20260924 00:59:03.381428 21875 log.cc:824] T 5d83f06ff1a044eb83bf454e274c3612 P 8a24b18d229349e6bf35c4bd079f41ca: Log is configured to *not* fsync() on all Append() calls
I20260924 00:59:03.382664 21874 tablet_bootstrap.cc:492] T 5d83f06ff1a044eb83bf454e274c3612 P 04eeeca62ebc4a8fa8cdb0e65be2dcc1: No bootstrap required, opened a new log
I20260924 00:59:03.383023 21874 ts_tablet_manager.cc:1397] T 5d83f06ff1a044eb83bf454e274c3612 P 04eeeca62ebc4a8fa8cdb0e65be2dcc1: Time spent bootstrapping tablet: real 0.023s user 0.014s sys 0.004s
I20260924 00:59:03.397455 21876 tablet_bootstrap.cc:492] T 5d83f06ff1a044eb83bf454e274c3612 P 6a6bd8bb8ee043268d1826c124777814: No bootstrap required, opened a new log
I20260924 00:59:03.397861 21876 ts_tablet_manager.cc:1397] T 5d83f06ff1a044eb83bf454e274c3612 P 6a6bd8bb8ee043268d1826c124777814: Time spent bootstrapping tablet: real 0.027s user 0.005s sys 0.006s
I20260924 00:59:03.413654 21874 raft_consensus.cc:348] T 5d83f06ff1a044eb83bf454e274c3612 P 04eeeca62ebc4a8fa8cdb0e65be2dcc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a6bd8bb8ee043268d1826c124777814" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 38481 } } peers { permanent_uuid: "8a24b18d229349e6bf35c4bd079f41ca" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46067 } } peers { permanent_uuid: "04eeeca62ebc4a8fa8cdb0e65be2dcc1" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 45571 } }
I20260924 00:59:03.416368 21876 raft_consensus.cc:348] T 5d83f06ff1a044eb83bf454e274c3612 P 6a6bd8bb8ee043268d1826c124777814 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a6bd8bb8ee043268d1826c124777814" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 38481 } } peers { permanent_uuid: "8a24b18d229349e6bf35c4bd079f41ca" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46067 } } peers { permanent_uuid: "04eeeca62ebc4a8fa8cdb0e65be2dcc1" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 45571 } }
I20260924 00:59:03.417052 21874 raft_consensus.cc:374] T 5d83f06ff1a044eb83bf454e274c3612 P 04eeeca62ebc4a8fa8cdb0e65be2dcc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:59:03.417600 21876 raft_consensus.cc:374] T 5d83f06ff1a044eb83bf454e274c3612 P 6a6bd8bb8ee043268d1826c124777814 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:59:03.417724 21876 raft_consensus.cc:729] T 5d83f06ff1a044eb83bf454e274c3612 P 6a6bd8bb8ee043268d1826c124777814 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a6bd8bb8ee043268d1826c124777814, State: Initialized, Role: FOLLOWER
I20260924 00:59:03.418465 21876 consensus_queue.cc:260] T 5d83f06ff1a044eb83bf454e274c3612 P 6a6bd8bb8ee043268d1826c124777814 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a6bd8bb8ee043268d1826c124777814" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 38481 } } peers { permanent_uuid: "8a24b18d229349e6bf35c4bd079f41ca" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46067 } } peers { permanent_uuid: "04eeeca62ebc4a8fa8cdb0e65be2dcc1" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 45571 } }
I20260924 00:59:03.418282 21874 raft_consensus.cc:729] T 5d83f06ff1a044eb83bf454e274c3612 P 04eeeca62ebc4a8fa8cdb0e65be2dcc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04eeeca62ebc4a8fa8cdb0e65be2dcc1, State: Initialized, Role: FOLLOWER
I20260924 00:59:03.421131 21874 consensus_queue.cc:260] T 5d83f06ff1a044eb83bf454e274c3612 P 04eeeca62ebc4a8fa8cdb0e65be2dcc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a6bd8bb8ee043268d1826c124777814" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 38481 } } peers { permanent_uuid: "8a24b18d229349e6bf35c4bd079f41ca" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46067 } } peers { permanent_uuid: "04eeeca62ebc4a8fa8cdb0e65be2dcc1" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 45571 } }
I20260924 00:59:03.422461 21875 tablet_bootstrap.cc:492] T 5d83f06ff1a044eb83bf454e274c3612 P 8a24b18d229349e6bf35c4bd079f41ca: No bootstrap required, opened a new log
I20260924 00:59:03.422842 21875 ts_tablet_manager.cc:1397] T 5d83f06ff1a044eb83bf454e274c3612 P 8a24b18d229349e6bf35c4bd079f41ca: Time spent bootstrapping tablet: real 0.053s user 0.020s sys 0.021s
I20260924 00:59:03.427302 21876 ts_tablet_manager.cc:1428] T 5d83f06ff1a044eb83bf454e274c3612 P 6a6bd8bb8ee043268d1826c124777814: Time spent starting tablet: real 0.029s user 0.023s sys 0.000s
I20260924 00:59:03.427754 21851 heartbeater.cc:503] Master 127.8.64.190:46357 was elected leader, sending a full tablet report...
I20260924 00:59:03.433653 21875 raft_consensus.cc:348] T 5d83f06ff1a044eb83bf454e274c3612 P 8a24b18d229349e6bf35c4bd079f41ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a6bd8bb8ee043268d1826c124777814" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 38481 } } peers { permanent_uuid: "8a24b18d229349e6bf35c4bd079f41ca" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46067 } } peers { permanent_uuid: "04eeeca62ebc4a8fa8cdb0e65be2dcc1" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 45571 } }
I20260924 00:59:03.434051 21572 heartbeater.cc:503] Master 127.8.64.190:46357 was elected leader, sending a full tablet report...
I20260924 00:59:03.435266 21874 ts_tablet_manager.cc:1428] T 5d83f06ff1a044eb83bf454e274c3612 P 04eeeca62ebc4a8fa8cdb0e65be2dcc1: Time spent starting tablet: real 0.052s user 0.013s sys 0.004s
I20260924 00:59:03.435626 21875 raft_consensus.cc:374] T 5d83f06ff1a044eb83bf454e274c3612 P 8a24b18d229349e6bf35c4bd079f41ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:59:03.435802 21875 raft_consensus.cc:729] T 5d83f06ff1a044eb83bf454e274c3612 P 8a24b18d229349e6bf35c4bd079f41ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a24b18d229349e6bf35c4bd079f41ca, State: Initialized, Role: FOLLOWER
I20260924 00:59:03.436556 21875 consensus_queue.cc:260] T 5d83f06ff1a044eb83bf454e274c3612 P 8a24b18d229349e6bf35c4bd079f41ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a6bd8bb8ee043268d1826c124777814" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 38481 } } peers { permanent_uuid: "8a24b18d229349e6bf35c4bd079f41ca" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46067 } } peers { permanent_uuid: "04eeeca62ebc4a8fa8cdb0e65be2dcc1" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 45571 } }
I20260924 00:59:03.440402 21717 heartbeater.cc:503] Master 127.8.64.190:46357 was elected leader, sending a full tablet report...
I20260924 00:59:03.440547 21875 ts_tablet_manager.cc:1428] T 5d83f06ff1a044eb83bf454e274c3612 P 8a24b18d229349e6bf35c4bd079f41ca: Time spent starting tablet: real 0.018s user 0.010s sys 0.005s
W20260924 00:59:03.506564 21718 tablet.cc:2369] T 5d83f06ff1a044eb83bf454e274c3612 P 8a24b18d229349e6bf35c4bd079f41ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:59:03.575997 21880 raft_consensus.cc:482] T 5d83f06ff1a044eb83bf454e274c3612 P 6a6bd8bb8ee043268d1826c124777814 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:59:03.576349 21880 raft_consensus.cc:504] T 5d83f06ff1a044eb83bf454e274c3612 P 6a6bd8bb8ee043268d1826c124777814 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a6bd8bb8ee043268d1826c124777814" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 38481 } } peers { permanent_uuid: "8a24b18d229349e6bf35c4bd079f41ca" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46067 } } peers { permanent_uuid: "04eeeca62ebc4a8fa8cdb0e65be2dcc1" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 45571 } }
I20260924 00:59:03.578127 21880 leader_election.cc:288] T 5d83f06ff1a044eb83bf454e274c3612 P 6a6bd8bb8ee043268d1826c124777814 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a24b18d229349e6bf35c4bd079f41ca (127.8.64.130:46067), 04eeeca62ebc4a8fa8cdb0e65be2dcc1 (127.8.64.131:45571)
W20260924 00:59:03.580137 21573 tablet.cc:2369] T 5d83f06ff1a044eb83bf454e274c3612 P 6a6bd8bb8ee043268d1826c124777814: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:59:03.585644 21872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:03.579241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:55681 (local address 127.8.64.131:45571)
0924 00:59:03.579420 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:03.579426 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:03.579874 (+ 448us) server_negotiation.cc:407] Connection header received
0924 00:59:03.580205 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:03.580212 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:03.580295 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:03.580376 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:59:03.581273 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:03.581959 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:03.583884 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:03.584696 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:03.584853 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:03.584926 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:03.585251 (+ 325us) server_negotiation.cc:299] Negotiation successful
0924 00:59:03.585381 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260924 00:59:03.585842 21891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:03.578674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.131:45571 (local address 127.8.64.129:55681)
0924 00:59:03.579761 (+ 1087us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:03.579777 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 00:59:03.580021 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:59:03.580629 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:59:03.580633 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:59:03.580659 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:59:03.581120 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:03.581130 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:03.582095 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:03.582102 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:03.583730 (+ 1628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:03.583742 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:03.584906 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:03.584912 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:03.585075 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:03.585111 (+ 36us) client_negotiation.cc:769] Sending connection context
0924 00:59:03.585210 (+ 99us) client_negotiation.cc:240] Negotiation successful
0924 00:59:03.585338 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":970,"mutex_wait_us":3,"thread_start_us":776,"threads_started":1}
I20260924 00:59:03.586912 21871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:03.578589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:34547 (local address 127.8.64.130:46067)
0924 00:59:03.578807 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:03.578815 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:03.579552 (+ 737us) server_negotiation.cc:407] Connection header received
0924 00:59:03.579898 (+ 346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:03.579907 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:03.580007 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:03.580116 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:59:03.581072 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:03.582183 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:03.584674 (+ 2491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:03.585807 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:03.585952 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:03.586042 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:03.586535 (+ 493us) server_negotiation.cc:299] Negotiation successful
0924 00:59:03.586658 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260924 00:59:03.586993 21890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:03.578553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.130:46067 (local address 127.8.64.129:34547)
0924 00:59:03.579417 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:03.579439 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0924 00:59:03.579681 (+ 242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:59:03.580293 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:59:03.580298 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:59:03.580325 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:59:03.580871 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:03.580889 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:03.582367 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:03.582373 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:03.584503 (+ 2130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:03.584517 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:03.586150 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:03.586156 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:03.586385 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:03.586407 (+ 22us) client_negotiation.cc:769] Sending connection context
0924 00:59:03.586508 (+ 101us) client_negotiation.cc:240] Negotiation successful
0924 00:59:03.586627 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":717,"mutex_wait_us":123,"thread_start_us":519,"threads_started":1}
I20260924 00:59:03.587071 21805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d83f06ff1a044eb83bf454e274c3612" candidate_uuid: "6a6bd8bb8ee043268d1826c124777814" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04eeeca62ebc4a8fa8cdb0e65be2dcc1" is_pre_election: true
I20260924 00:59:03.587651 21805 raft_consensus.cc:2454] T 5d83f06ff1a044eb83bf454e274c3612 P 04eeeca62ebc4a8fa8cdb0e65be2dcc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a6bd8bb8ee043268d1826c124777814 in term 0.
I20260924 00:59:03.587896 21671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d83f06ff1a044eb83bf454e274c3612" candidate_uuid: "6a6bd8bb8ee043268d1826c124777814" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a24b18d229349e6bf35c4bd079f41ca" is_pre_election: true
I20260924 00:59:03.588371 21459 leader_election.cc:302] T 5d83f06ff1a044eb83bf454e274c3612 P 6a6bd8bb8ee043268d1826c124777814 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04eeeca62ebc4a8fa8cdb0e65be2dcc1, 6a6bd8bb8ee043268d1826c124777814; no voters:
I20260924 00:59:03.588447 21671 raft_consensus.cc:2454] T 5d83f06ff1a044eb83bf454e274c3612 P 8a24b18d229349e6bf35c4bd079f41ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a6bd8bb8ee043268d1826c124777814 in term 0.
I20260924 00:59:03.588840 21880 raft_consensus.cc:2781] T 5d83f06ff1a044eb83bf454e274c3612 P 6a6bd8bb8ee043268d1826c124777814 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:59:03.588992 21880 raft_consensus.cc:482] T 5d83f06ff1a044eb83bf454e274c3612 P 6a6bd8bb8ee043268d1826c124777814 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:59:03.589109 21880 raft_consensus.cc:3037] T 5d83f06ff1a044eb83bf454e274c3612 P 6a6bd8bb8ee043268d1826c124777814 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:59:03.591339 21880 raft_consensus.cc:504] T 5d83f06ff1a044eb83bf454e274c3612 P 6a6bd8bb8ee043268d1826c124777814 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a6bd8bb8ee043268d1826c124777814" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 38481 } } peers { permanent_uuid: "8a24b18d229349e6bf35c4bd079f41ca" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46067 } } peers { permanent_uuid: "04eeeca62ebc4a8fa8cdb0e65be2dcc1" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 45571 } }
I20260924 00:59:03.591980 21880 leader_election.cc:288] T 5d83f06ff1a044eb83bf454e274c3612 P 6a6bd8bb8ee043268d1826c124777814 [CANDIDATE]: Term 1 election: Requested vote from peers 8a24b18d229349e6bf35c4bd079f41ca (127.8.64.130:46067), 04eeeca62ebc4a8fa8cdb0e65be2dcc1 (127.8.64.131:45571)
I20260924 00:59:03.592402 21671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d83f06ff1a044eb83bf454e274c3612" candidate_uuid: "6a6bd8bb8ee043268d1826c124777814" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a24b18d229349e6bf35c4bd079f41ca"
I20260924 00:59:03.592496 21805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d83f06ff1a044eb83bf454e274c3612" candidate_uuid: "6a6bd8bb8ee043268d1826c124777814" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04eeeca62ebc4a8fa8cdb0e65be2dcc1"
I20260924 00:59:03.592612 21671 raft_consensus.cc:3037] T 5d83f06ff1a044eb83bf454e274c3612 P 8a24b18d229349e6bf35c4bd079f41ca [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:59:03.592722 21805 raft_consensus.cc:3037] T 5d83f06ff1a044eb83bf454e274c3612 P 04eeeca62ebc4a8fa8cdb0e65be2dcc1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:59:03.595021 21805 raft_consensus.cc:2454] T 5d83f06ff1a044eb83bf454e274c3612 P 04eeeca62ebc4a8fa8cdb0e65be2dcc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a6bd8bb8ee043268d1826c124777814 in term 1.
I20260924 00:59:03.595419 21459 leader_election.cc:302] T 5d83f06ff1a044eb83bf454e274c3612 P 6a6bd8bb8ee043268d1826c124777814 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04eeeca62ebc4a8fa8cdb0e65be2dcc1, 6a6bd8bb8ee043268d1826c124777814; no voters:
I20260924 00:59:03.595715 21880 raft_consensus.cc:2781] T 5d83f06ff1a044eb83bf454e274c3612 P 6a6bd8bb8ee043268d1826c124777814 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:59:03.595781 21671 raft_consensus.cc:2454] T 5d83f06ff1a044eb83bf454e274c3612 P 8a24b18d229349e6bf35c4bd079f41ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a6bd8bb8ee043268d1826c124777814 in term 1.
I20260924 00:59:03.596701 21880 raft_consensus.cc:686] T 5d83f06ff1a044eb83bf454e274c3612 P 6a6bd8bb8ee043268d1826c124777814 [term 1 LEADER]: Becoming Leader. State: Replica: 6a6bd8bb8ee043268d1826c124777814, State: Running, Role: LEADER
I20260924 00:59:03.597173 21880 consensus_queue.cc:237] T 5d83f06ff1a044eb83bf454e274c3612 P 6a6bd8bb8ee043268d1826c124777814 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a6bd8bb8ee043268d1826c124777814" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 38481 } } peers { permanent_uuid: "8a24b18d229349e6bf35c4bd079f41ca" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46067 } } peers { permanent_uuid: "04eeeca62ebc4a8fa8cdb0e65be2dcc1" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 45571 } }
I20260924 00:59:03.602607 21372 catalog_manager.cc:5495] T 5d83f06ff1a044eb83bf454e274c3612 P 6a6bd8bb8ee043268d1826c124777814 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a6bd8bb8ee043268d1826c124777814 (127.8.64.129). New cstate: current_term: 1 leader_uuid: "6a6bd8bb8ee043268d1826c124777814" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a6bd8bb8ee043268d1826c124777814" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 38481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a24b18d229349e6bf35c4bd079f41ca" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "04eeeca62ebc4a8fa8cdb0e65be2dcc1" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 45571 } health_report { overall_health: UNKNOWN } } }
W20260924 00:59:03.643923 21852 tablet.cc:2369] T 5d83f06ff1a044eb83bf454e274c3612 P 04eeeca62ebc4a8fa8cdb0e65be2dcc1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:59:03.687191 21870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:03.681539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58950 (local address 127.8.64.129:38481)
0924 00:59:03.681752 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:03.681759 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:03.681803 (+ 44us) server_negotiation.cc:407] Connection header received
0924 00:59:03.682084 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:03.682096 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:03.682199 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:03.682382 (+ 183us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:59:03.682760 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:03.683511 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:03.684934 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:03.685345 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:03.685484 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:03.686012 (+ 528us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:59:03.686594 (+ 582us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:59:03.686755 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:03.686834 (+ 79us) server_negotiation.cc:299] Negotiation successful
0924 00:59:03.686913 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260924 00:59:03.695575 21671 raft_consensus.cc:1261] T 5d83f06ff1a044eb83bf454e274c3612 P 8a24b18d229349e6bf35c4bd079f41ca [term 1 FOLLOWER]: Refusing update from remote peer 6a6bd8bb8ee043268d1826c124777814: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:59:03.695575 21805 raft_consensus.cc:1261] T 5d83f06ff1a044eb83bf454e274c3612 P 04eeeca62ebc4a8fa8cdb0e65be2dcc1 [term 1 FOLLOWER]: Refusing update from remote peer 6a6bd8bb8ee043268d1826c124777814: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:59:03.696581 21880 consensus_queue.cc:1035] T 5d83f06ff1a044eb83bf454e274c3612 P 6a6bd8bb8ee043268d1826c124777814 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a24b18d229349e6bf35c4bd079f41ca" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 46067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:59:03.696902 21892 consensus_queue.cc:1035] T 5d83f06ff1a044eb83bf454e274c3612 P 6a6bd8bb8ee043268d1826c124777814 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04eeeca62ebc4a8fa8cdb0e65be2dcc1" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 45571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:59:03.713932 21897 mvcc.cc:205] Tried to move back new op lower bound from 7332706482965942272 to 7332706482578100224. Current Snapshot: MvccSnapshot[applied={T|T < 7332706482965942272}]
I20260924 00:59:03.745055 21372 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:59:03.745352 21372 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:59:03.749653 21372 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ca0f128d53134d24a09ce3442e33f822: Sending DeleteTablet for 3 replicas of tablet 5d83f06ff1a044eb83bf454e274c3612
I20260924 00:59:03.750907 21506 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d83f06ff1a044eb83bf454e274c3612 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:59:03 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:58934
I20260924 00:59:03.751058 21651 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d83f06ff1a044eb83bf454e274c3612 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:59:03 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:44036
I20260924 00:59:03.751322 21785 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d83f06ff1a044eb83bf454e274c3612 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:59:03 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:55786
I20260924 00:59:03.758625 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 21411
I20260924 00:59:03.791261 21904 tablet_replica.cc:330] T 5d83f06ff1a044eb83bf454e274c3612 P 6a6bd8bb8ee043268d1826c124777814: stopping tablet replica
I20260924 00:59:03.791940 21906 tablet_replica.cc:330] T 5d83f06ff1a044eb83bf454e274c3612 P 04eeeca62ebc4a8fa8cdb0e65be2dcc1: stopping tablet replica
I20260924 00:59:03.794718 21904 raft_consensus.cc:2229] T 5d83f06ff1a044eb83bf454e274c3612 P 6a6bd8bb8ee043268d1826c124777814 [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:59:03.795403 21906 raft_consensus.cc:2229] T 5d83f06ff1a044eb83bf454e274c3612 P 04eeeca62ebc4a8fa8cdb0e65be2dcc1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:59:03.797574 21906 raft_consensus.cc:2258] T 5d83f06ff1a044eb83bf454e274c3612 P 04eeeca62ebc4a8fa8cdb0e65be2dcc1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:59:03.798768 21904 raft_consensus.cc:2258] T 5d83f06ff1a044eb83bf454e274c3612 P 6a6bd8bb8ee043268d1826c124777814 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:59:03.801424 21906 ts_tablet_manager.cc:1905] T 5d83f06ff1a044eb83bf454e274c3612 P 04eeeca62ebc4a8fa8cdb0e65be2dcc1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:59:03.803912 21904 ts_tablet_manager.cc:1905] T 5d83f06ff1a044eb83bf454e274c3612 P 6a6bd8bb8ee043268d1826c124777814: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:59:03.805039 21905 tablet_replica.cc:330] T 5d83f06ff1a044eb83bf454e274c3612 P 8a24b18d229349e6bf35c4bd079f41ca: stopping tablet replica
I20260924 00:59:03.810813 21905 raft_consensus.cc:2229] T 5d83f06ff1a044eb83bf454e274c3612 P 8a24b18d229349e6bf35c4bd079f41ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:59:03.811394 21905 raft_consensus.cc:2258] T 5d83f06ff1a044eb83bf454e274c3612 P 8a24b18d229349e6bf35c4bd079f41ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:59:03.811972 21906 ts_tablet_manager.cc:1918] T 5d83f06ff1a044eb83bf454e274c3612 P 04eeeca62ebc4a8fa8cdb0e65be2dcc1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:59:03.812155 21906 log.cc:1196] T 5d83f06ff1a044eb83bf454e274c3612 P 04eeeca62ebc4a8fa8cdb0e65be2dcc1: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-2/wal/wals/5d83f06ff1a044eb83bf454e274c3612
I20260924 00:59:03.812731 21906 ts_tablet_manager.cc:1939] T 5d83f06ff1a044eb83bf454e274c3612 P 04eeeca62ebc4a8fa8cdb0e65be2dcc1: Deleting consensus metadata
I20260924 00:59:03.814584 21357 catalog_manager.cc:4841] TS 04eeeca62ebc4a8fa8cdb0e65be2dcc1 (127.8.64.131:45571): tablet 5d83f06ff1a044eb83bf454e274c3612 (table test-table [id=b12f584c38434db5b871849557c570e5]) successfully deleted
I20260924 00:59:03.816191 21905 ts_tablet_manager.cc:1905] T 5d83f06ff1a044eb83bf454e274c3612 P 8a24b18d229349e6bf35c4bd079f41ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:59:03.817198 21870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:03.759817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58958 (local address 127.8.64.129:38481)
0924 00:59:03.760019 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:03.760024 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:03.784152 (+ 24128us) server_negotiation.cc:407] Connection header received
0924 00:59:03.784638 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:03.784645 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:03.784743 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:03.784837 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:03.786919 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:03.788089 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:03.792257 (+ 4168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:03.792885 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:03.792981 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:03.805358 (+ 12377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:03.805371 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:03.805389 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:03.805431 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:03.809388 (+ 3957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:03.811621 (+ 2233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:03.811630 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:03.811634 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:03.811799 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:03.812424 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:03.812429 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:03.812432 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:03.812964 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:03.813267 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:03.816571 (+ 3304us) server_negotiation.cc:299] Negotiation successful
0924 00:59:03.816837 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260924 00:59:03.819041 21567 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:59:03.826674 21905 ts_tablet_manager.cc:1918] T 5d83f06ff1a044eb83bf454e274c3612 P 8a24b18d229349e6bf35c4bd079f41ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:59:03.826897 21905 log.cc:1196] T 5d83f06ff1a044eb83bf454e274c3612 P 8a24b18d229349e6bf35c4bd079f41ca: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-1/wal/wals/5d83f06ff1a044eb83bf454e274c3612
I20260924 00:59:03.827445 21905 ts_tablet_manager.cc:1939] T 5d83f06ff1a044eb83bf454e274c3612 P 8a24b18d229349e6bf35c4bd079f41ca: Deleting consensus metadata
I20260924 00:59:03.829377 21359 catalog_manager.cc:4841] TS 8a24b18d229349e6bf35c4bd079f41ca (127.8.64.130:46067): tablet 5d83f06ff1a044eb83bf454e274c3612 (table test-table [id=b12f584c38434db5b871849557c570e5]) successfully deleted
I20260924 00:59:03.921631 21904 ts_tablet_manager.cc:1918] T 5d83f06ff1a044eb83bf454e274c3612 P 6a6bd8bb8ee043268d1826c124777814: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:59:03.921877 21904 log.cc:1196] T 5d83f06ff1a044eb83bf454e274c3612 P 6a6bd8bb8ee043268d1826c124777814: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790211343099518-8450-0/minicluster-data/ts-0/wal/wals/5d83f06ff1a044eb83bf454e274c3612
I20260924 00:59:03.922555 21904 ts_tablet_manager.cc:1939] T 5d83f06ff1a044eb83bf454e274c3612 P 6a6bd8bb8ee043268d1826c124777814: Deleting consensus metadata
I20260924 00:59:03.924219 21357 catalog_manager.cc:4841] TS 6a6bd8bb8ee043268d1826c124777814 (127.8.64.129:38481): tablet 5d83f06ff1a044eb83bf454e274c3612 (table test-table [id=b12f584c38434db5b871849557c570e5]) successfully deleted
I20260924 00:59:03.926620 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 21411
I20260924 00:59:03.945183 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 21576
I20260924 00:59:03.958391 21871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:03.946006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44040 (local address 127.8.64.130:46067)
0924 00:59:03.946208 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:03.946214 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:03.946346 (+ 132us) server_negotiation.cc:407] Connection header received
0924 00:59:03.946589 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:03.946594 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:03.946656 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:03.946777 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:03.948016 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:03.948728 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:03.950010 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:03.950492 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:03.950630 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:03.953015 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:03.953027 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:03.953042 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:03.953080 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:03.955780 (+ 2700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:03.956413 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:03.956420 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:03.956423 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:03.956560 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:03.956932 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:03.956937 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:03.956939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:03.957307 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:03.957527 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:03.957774 (+ 247us) server_negotiation.cc:299] Negotiation successful
0924 00:59:03.957932 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260924 00:59:03.959097 21712 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:59:04.047811 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 21576
I20260924 00:59:04.064325 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 21721
I20260924 00:59:04.078930 21872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:04.065145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55796 (local address 127.8.64.131:45571)
0924 00:59:04.065326 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:04.065330 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:04.065371 (+ 41us) server_negotiation.cc:407] Connection header received
0924 00:59:04.065495 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:04.065499 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:04.065565 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:04.065654 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:04.067089 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:04.067831 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:04.069209 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:04.069592 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:04.069702 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:04.072412 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:04.072425 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:04.072441 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:04.072479 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:04.075590 (+ 3111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:04.076483 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:04.076506 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:04.076513 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:04.076711 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:04.077197 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:04.077204 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:04.077210 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:04.077711 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:04.078022 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:04.078350 (+ 328us) server_negotiation.cc:299] Negotiation successful
0924 00:59:04.078558 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260924 00:59:04.079535 21846 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:59:04.168789 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 21721
I20260924 00:59:04.185166 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 21340
I20260924 00:59:04.186550 21403 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:59:04.299204 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 21340
I20260924 00:59:04.321314 8450 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260924 00:59:04.656456 8450 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260924 00:59:04.656708 8450 mini_ranger.cc:66] Stopping Ranger...
I20260924 00:59:05.001827 8450 mini_ranger.cc:68] Stopped Ranger
2026-09-24 00:59:05.002 UTC [20386] LOG: received smart shutdown request
2026-09-24 00:59:05.005 UTC [20386] LOG: background worker "logical replication launcher" (PID 20395) exited with exit code 1
2026-09-24 00:59:05.006 UTC [20390] LOG: shutting down
2026-09-24 00:59:05.206 UTC [20386] LOG: database system is shut down
2026-09-24T00:59:05Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (51769 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:59:05 dist-test-slave-mmhs krb5kdc[21920](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:59:05 dist-test-slave-mmhs krb5kdc[21920](info): set up 2 sockets
Sep 24 00:59:05 dist-test-slave-mmhs krb5kdc[21920](info): commencing operation
krb5kdc: starting...
W20260924 00:59:06.452308 8450 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.002s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Sep 24 00:59:06 dist-test-slave-mmhs krb5kdc[21920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211546, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:59:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:59:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.64.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
I20260924 00:59:06.687896 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:36169
--webserver_interface=127.8.64.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:33379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:36169,127.8.64.189:43831,127.8.64.188:42459
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:59:07.018682 21934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:59:07.019018 21934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:59:07.019201 21934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:59:07.026564 21934 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:59:07.026705 21934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:59:07.026814 21934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:59:07.026921 21934 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:59:07.027062 21934 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:59:07.037657 21934 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:33379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:36169,127.8.64.189:43831,127.8.64.188:42459
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:36169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:59:07.039100 21934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:59:07.040395 21934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:59:07.052253 21939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:59:07.052292 21942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:59:07.052330 21940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:59:07.053128 21934 server_base.cc:1034] running on GCE node
I20260924 00:59:07.054045 21934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:59:07.055508 21934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:59:07.056763 21934 hybrid_clock.cc:648] HybridClock initialized: now 1790211547056721 us; error 54 us; skew 500 ppm
I20260924 00:59:07.057173 21934 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:59:07 dist-test-slave-mmhs krb5kdc[21920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211547, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:59:07.065585 21934 init.cc:377] Logged in from keytab as kudu/127.8.64.190@KRBTEST.COM (short username kudu)
I20260924 00:59:07.067211 21934 webserver.cc:460] Webserver started at http://127.8.64.190:34923/ using document root <none> and password file <none>
I20260924 00:59:07.067932 21934 fs_manager.cc:359] Metadata directory not provided
I20260924 00:59:07.068082 21934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:59:07.068392 21934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:59:07.071504 21934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "3580661dd79443dfb823fc61af395a90"
format_stamp: "Formatted at 2026-09-24 00:59:07 on dist-test-slave-mmhs"
server_key: "42231f94055337a4efecca83f8c83acb"
server_key_iv: "13ee50a9e927421fe30d626505e7cc80"
server_key_version: "encryptionkey@0"
I20260924 00:59:07.072330 21934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "3580661dd79443dfb823fc61af395a90"
format_stamp: "Formatted at 2026-09-24 00:59:07 on dist-test-slave-mmhs"
server_key: "42231f94055337a4efecca83f8c83acb"
server_key_iv: "13ee50a9e927421fe30d626505e7cc80"
server_key_version: "encryptionkey@0"
I20260924 00:59:07.077540 21934 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260924 00:59:07.081064 21949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:07.082430 21934 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 00:59:07.082609 21934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "3580661dd79443dfb823fc61af395a90"
format_stamp: "Formatted at 2026-09-24 00:59:07 on dist-test-slave-mmhs"
server_key: "42231f94055337a4efecca83f8c83acb"
server_key_iv: "13ee50a9e927421fe30d626505e7cc80"
server_key_version: "encryptionkey@0"
I20260924 00:59:07.082808 21934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:59:07.097906 21934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:59:07.102519 21934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:59:07.102813 21934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:59:07.121031 21934 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.190:36169
I20260924 00:59:07.121066 22000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.190:36169 every 8 connection(s)
I20260924 00:59:07.122468 21934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 00:59:07.126185 22001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:59:07.129813 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 21934
I20260924 00:59:07.130025 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 00:59:07.130774 8450 external_mini_cluster.cc:1428] Setting key 680935be2f791d8ec5c6e0a9d2e210e1
I20260924 00:59:07.132753 22001 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.64.190" port: 36169 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 00:59:07 dist-test-slave-mmhs krb5kdc[21920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211547, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.64.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.189@KRBTEST.COM" created.
I20260924 00:59:07.157764 22004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:07.135924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54386 (local address 127.8.64.190:36169)
0924 00:59:07.138033 (+ 2109us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:07.138047 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:07.138106 (+ 59us) server_negotiation.cc:407] Connection header received
0924 00:59:07.139806 (+ 1700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:07.139814 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:07.140134 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:07.140306 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:07.141977 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:07.143283 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:07.145694 (+ 2411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:07.146258 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:07.146464 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:07.150561 (+ 4097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:07.150569 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:07.150580 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:07.150608 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:07.153406 (+ 2798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:07.154082 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:07.154097 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:07.154101 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:07.154241 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:07.154683 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:07.154690 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:07.154693 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:07.155173 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:07.155391 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:07.155905 (+ 514us) server_negotiation.cc:299] Negotiation successful
0924 00:59:07.156328 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":1873,"spinlock_wait_cycles":56576,"thread_start_us":1619,"threads_started":1}
I20260924 00:59:07.157787 22003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:07.135924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:36169 (local address 127.0.0.1:54386)
0924 00:59:07.136881 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:07.136931 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0924 00:59:07.139329 (+ 2398us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:59:07.140405 (+ 1076us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:59:07.140416 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:59:07.141230 (+ 814us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:59:07.141767 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:07.141801 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:07.143453 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:07.143461 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:07.145468 (+ 2007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:07.145486 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:07.146547 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:07.146556 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:07.146788 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:07.147483 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:59:07.147517 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:59:07.150284 (+ 2767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:59:07.153534 (+ 3250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:59:07.153546 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:59:07.153558 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:59:07.153951 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:59:07.154414 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:59:07.154419 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:59:07.154423 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:59:07.154574 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:59:07.155331 (+ 757us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:59:07.155340 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:59:07.155541 (+ 201us) client_negotiation.cc:769] Sending connection context
0924 00:59:07.155791 (+ 250us) client_negotiation.cc:240] Negotiation successful
0924 00:59:07.156448 (+ 657us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"mutex_wait_us":189,"thread_start_us":403,"threads_started":1}
I20260924 00:59:07.160934 22001 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43831 } has no permanent_uuid. Determining permanent_uuid...
I20260924 00:59:07.162147 22003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:07.161765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:43831 (local address 127.0.0.1:49432)
0924 00:59:07.161929 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:07.162033 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.189:43831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260924 00:59:07.162463 21951 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.64.189:43831: connect: Connection refused (error 111)
I20260924 00:59:07.164045 22003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:07.163715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:43831 (local address 127.0.0.1:49442)
0924 00:59:07.163853 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:07.163919 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.189:43831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260924 00:59:07.164311 22001 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.64.189:43831: Network error: Client connection negotiation failed: client connection to 127.8.64.189:43831: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.189.keytab.
Entry for principal kudu/127.8.64.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.189.keytab.
I20260924 00:59:07.186086 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.189:43831
--webserver_interface=127.8.64.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.189.keytab
--principal=kudu/127.8.64.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:33379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:36169,127.8.64.189:43831,127.8.64.188:42459
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 00:59:07.213629 22001 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43831 } attempt: 1
I20260924 00:59:07.214790 22003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:07.214377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:43831 (local address 127.0.0.1:49448)
0924 00:59:07.214562 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:07.214651 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.189:43831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260924 00:59:07.215806 22003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:07.215486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:43831 (local address 127.0.0.1:49450)
0924 00:59:07.215617 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:07.215686 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.189:43831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260924 00:59:07.216044 22001 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.64.189:43831: Network error: Client connection negotiation failed: client connection to 127.8.64.189:43831: connect: Connection refused (error 111)
I20260924 00:59:07.284291 22001 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43831 } attempt: 2
I20260924 00:59:07.285504 22003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:07.285024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:43831 (local address 127.0.0.1:49456)
0924 00:59:07.285283 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:07.285373 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.189:43831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260924 00:59:07.286643 22003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:07.286242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:43831 (local address 127.0.0.1:49468)
0924 00:59:07.286411 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:07.286484 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.189:43831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260924 00:59:07.286881 22001 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.64.189:43831: Network error: Client connection negotiation failed: client connection to 127.8.64.189:43831: connect: Connection refused (error 111)
I20260924 00:59:07.378150 22001 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43831 } attempt: 3
I20260924 00:59:07.379405 22003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:07.378961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:43831 (local address 127.0.0.1:49476)
0924 00:59:07.379168 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:07.379272 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.189:43831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260924 00:59:07.380439 22003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:07.380112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:43831 (local address 127.0.0.1:49490)
0924 00:59:07.380250 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:07.380316 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.189:43831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260924 00:59:07.380649 22001 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.64.189:43831: Network error: Client connection negotiation failed: client connection to 127.8.64.189:43831: connect: Connection refused (error 111)
W20260924 00:59:07.513199 22008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:59:07.513484 22008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:59:07.513574 22008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:59:07.520680 22008 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:59:07.520784 22008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:59:07.520831 22008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:59:07.520884 22008 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:59:07.520931 22008 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:59:07.523898 22001 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43831 } attempt: 4
I20260924 00:59:07.525152 22003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:07.524574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:43831 (local address 127.0.0.1:49504)
0924 00:59:07.524905 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:07.525008 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.189:43831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191}
I20260924 00:59:07.526365 22003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:07.525893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:43831 (local address 127.0.0.1:49520)
0924 00:59:07.526087 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:07.526183 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.189:43831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260924 00:59:07.526634 22001 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.64.189:43831: Network error: Client connection negotiation failed: client connection to 127.8.64.189:43831: connect: Connection refused (error 111)
I20260924 00:59:07.531522 22008 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:33379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:36169,127.8.64.189:43831,127.8.64.188:42459
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.189.keytab
--principal=kudu/127.8.64.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.189:43831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.64.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:59:07.532886 22008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:59:07.534220 22008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:59:07.546386 22014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:59:07.546468 22016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:59:07.546450 22013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:59:07.547664 22008 server_base.cc:1034] running on GCE node
I20260924 00:59:07.548491 22008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:59:07.549947 22008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:59:07.551179 22008 hybrid_clock.cc:648] HybridClock initialized: now 1790211547551133 us; error 39 us; skew 500 ppm
I20260924 00:59:07.551537 22008 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:59:07 dist-test-slave-mmhs krb5kdc[21920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211547, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:59:07.554801 22008 init.cc:377] Logged in from keytab as kudu/127.8.64.189@KRBTEST.COM (short username kudu)
I20260924 00:59:07.556304 22008 webserver.cc:460] Webserver started at http://127.8.64.189:34751/ using document root <none> and password file <none>
I20260924 00:59:07.557049 22008 fs_manager.cc:359] Metadata directory not provided
I20260924 00:59:07.557199 22008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:59:07.557507 22008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:59:07.560547 22008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-1/data/instance:
uuid: "6958b0521072447f8df5b1db8a3ef826"
format_stamp: "Formatted at 2026-09-24 00:59:07 on dist-test-slave-mmhs"
server_key: "ad728648ebf38eff4b794bf88f6f60dd"
server_key_iv: "55ae8323b7914121b03cab22256fe9b8"
server_key_version: "encryptionkey@0"
I20260924 00:59:07.561372 22008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-1/wal/instance:
uuid: "6958b0521072447f8df5b1db8a3ef826"
format_stamp: "Formatted at 2026-09-24 00:59:07 on dist-test-slave-mmhs"
server_key: "ad728648ebf38eff4b794bf88f6f60dd"
server_key_iv: "55ae8323b7914121b03cab22256fe9b8"
server_key_version: "encryptionkey@0"
I20260924 00:59:07.566536 22008 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260924 00:59:07.569985 22023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:07.571385 22008 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 00:59:07.571581 22008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-1/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-1/wal
uuid: "6958b0521072447f8df5b1db8a3ef826"
format_stamp: "Formatted at 2026-09-24 00:59:07 on dist-test-slave-mmhs"
server_key: "ad728648ebf38eff4b794bf88f6f60dd"
server_key_iv: "55ae8323b7914121b03cab22256fe9b8"
server_key_version: "encryptionkey@0"
I20260924 00:59:07.571800 22008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:59:07.618341 22008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:59:07.622941 22008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:59:07.623255 22008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:59:07.641746 22008 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.189:43831
I20260924 00:59:07.641780 22075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.189:43831 every 8 connection(s)
I20260924 00:59:07.643172 22008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-1/data/info.pb
I20260924 00:59:07.646874 22076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:59:07.650679 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 22008
I20260924 00:59:07.650867 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-1/wal/instance
I20260924 00:59:07.651304 8450 external_mini_cluster.cc:1428] Setting key 8758ac62c1d9a4d5615361d2a5454af7
I20260924 00:59:07.653389 22076 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.64.190" port: 36169 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 00:59:07 dist-test-slave-mmhs krb5kdc[21920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211547, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.189@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.64.188@KRBTEST.COM; defaulting to no policy
I20260924 00:59:07.675117 22004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:07.656107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54396 (local address 127.8.64.190:36169)
0924 00:59:07.656307 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:07.656312 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:07.657660 (+ 1348us) server_negotiation.cc:407] Connection header received
0924 00:59:07.660273 (+ 2613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:07.660279 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:07.660378 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:07.660486 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:07.661958 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:07.663148 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:07.664677 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:07.665492 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:07.665641 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:07.669203 (+ 3562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:07.669208 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:07.669211 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:07.669244 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:07.671975 (+ 2731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:07.672715 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:07.672723 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:07.672728 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:07.672856 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:07.673280 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:07.673287 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:07.673290 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:07.673574 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:07.673750 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:07.674502 (+ 752us) server_negotiation.cc:299] Negotiation successful
0924 00:59:07.674737 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.188@KRBTEST.COM" created.
I20260924 00:59:07.676479 22078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:07.656704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:36169 (local address 127.0.0.1:54396)
0924 00:59:07.657477 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:07.657515 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0924 00:59:07.659986 (+ 2471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:59:07.660710 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:59:07.660718 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:59:07.661217 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:59:07.661777 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:07.661793 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:07.663332 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:07.663335 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:07.664528 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:07.664540 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:07.665702 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:07.665706 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:07.665879 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:07.666703 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:59:07.666726 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:59:07.668942 (+ 2216us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:59:07.672128 (+ 3186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:59:07.672137 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:59:07.672154 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:59:07.672580 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:59:07.672970 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:59:07.672975 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:59:07.672979 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:59:07.673168 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:59:07.673750 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:59:07.673759 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:59:07.674091 (+ 332us) client_negotiation.cc:769] Sending connection context
0924 00:59:07.674414 (+ 323us) client_negotiation.cc:240] Negotiation successful
0924 00:59:07.675033 (+ 619us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":345,"threads_started":1}
I20260924 00:59:07.678287 22076 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43831 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 00:59:07 dist-test-slave-mmhs krb5kdc[21920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211547, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.189@KRBTEST.COM for kudu/127.8.64.189@KRBTEST.COM
I20260924 00:59:07.695394 22078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:07.679212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:43831 (local address 127.0.0.1:49530)
0924 00:59:07.679428 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:07.679446 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0924 00:59:07.679557 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:59:07.680626 (+ 1069us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:59:07.680629 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:59:07.680997 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:59:07.681129 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:07.681137 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:07.683165 (+ 2028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:07.683169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:07.684181 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:07.684198 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:07.685055 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:07.685060 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:07.685230 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:07.685844 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:59:07.685872 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:59:07.688072 (+ 2200us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:59:07.692666 (+ 4594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:59:07.692670 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:59:07.692673 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:59:07.693063 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:59:07.693550 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:59:07.693554 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:59:07.693558 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:59:07.693735 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:59:07.694602 (+ 867us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:59:07.694605 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:59:07.694715 (+ 110us) client_negotiation.cc:769] Sending connection context
0924 00:59:07.694832 (+ 117us) client_negotiation.cc:240] Negotiation successful
0924 00:59:07.695116 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260924 00:59:07.695802 22080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:07.679338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49530 (local address 127.8.64.189:43831)
0924 00:59:07.680104 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:07.680114 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:07.680147 (+ 33us) server_negotiation.cc:407] Connection header received
0924 00:59:07.680207 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:07.680217 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:07.680463 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:07.680619 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:07.681289 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:07.683029 (+ 1740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:07.684322 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:07.684861 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:07.684957 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:07.688294 (+ 3337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:07.688310 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:07.688329 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:07.688370 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:07.692512 (+ 4142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:07.693205 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:07.693217 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:07.693223 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:07.693391 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:07.693878 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:07.693886 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:07.693891 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:07.694401 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:07.694622 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:07.695022 (+ 400us) server_negotiation.cc:299] Negotiation successful
0924 00:59:07.695320 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":408,"threads_started":1}
I20260924 00:59:07.698248 22076 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.64.188" port: 42459 } has no permanent_uuid. Determining permanent_uuid...
I20260924 00:59:07.699340 22078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:07.698901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:42459 (local address 127.0.0.1:43374)
0924 00:59:07.699086 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:07.699211 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:42459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260924 00:59:07.699568 22024 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.64.188:42459: connect: Connection refused (error 111)
I20260924 00:59:07.700985 22078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:07.700666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:42459 (local address 127.0.0.1:43382)
0924 00:59:07.700808 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:07.700891 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:42459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260924 00:59:07.701215 22076 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.64.188:42459: Network error: Client connection negotiation failed: client connection to 127.8.64.188:42459: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.188.keytab.
Entry for principal kudu/127.8.64.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.188.keytab.
I20260924 00:59:07.710886 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.188:42459
--webserver_interface=127.8.64.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.188.keytab
--principal=kudu/127.8.64.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:33379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:36169,127.8.64.189:43831,127.8.64.188:42459
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 00:59:07.750545 22076 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.64.188" port: 42459 } attempt: 1
I20260924 00:59:07.751772 22078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:07.751312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:42459 (local address 127.0.0.1:43390)
0924 00:59:07.751505 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:07.751615 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:42459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260924 00:59:07.752835 22078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:07.752494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:42459 (local address 127.0.0.1:43400)
0924 00:59:07.752645 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:07.752717 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:42459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260924 00:59:07.753127 22076 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.64.188:42459: Network error: Client connection negotiation failed: client connection to 127.8.64.188:42459: connect: Connection refused (error 111)
I20260924 00:59:07.821400 22076 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.64.188" port: 42459 } attempt: 2
I20260924 00:59:07.822714 22078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:07.822122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:42459 (local address 127.0.0.1:43412)
0924 00:59:07.822394 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:07.822505 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:42459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260924 00:59:07.823865 22078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:07.823467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:42459 (local address 127.0.0.1:43424)
0924 00:59:07.823631 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:07.823730 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:42459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260924 00:59:07.824090 22076 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.64.188:42459: Network error: Client connection negotiation failed: client connection to 127.8.64.188:42459: connect: Connection refused (error 111)
I20260924 00:59:07.825889 22001 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43831 } attempt: 5
Sep 24 00:59:07 dist-test-slave-mmhs krb5kdc[21920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211547, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for kudu/127.8.64.189@KRBTEST.COM
I20260924 00:59:07.838815 22080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:07.826569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49546 (local address 127.8.64.189:43831)
0924 00:59:07.826704 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:07.826710 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:07.826789 (+ 79us) server_negotiation.cc:407] Connection header received
0924 00:59:07.827008 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:07.827014 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:07.827096 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:07.827208 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:07.828060 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:07.828758 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:07.829914 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:07.830447 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:07.830571 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:07.833592 (+ 3021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:07.833602 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:07.833605 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:07.833636 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:07.836236 (+ 2600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:07.836898 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:07.836905 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:07.836915 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:07.837030 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:07.837399 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:07.837405 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:07.837411 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:07.837685 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:07.837854 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:07.838251 (+ 397us) server_negotiation.cc:299] Negotiation successful
0924 00:59:07.838484 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260924 00:59:07.838819 22003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:07.826509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:43831 (local address 127.0.0.1:49546)
0924 00:59:07.826714 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:07.826733 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0924 00:59:07.826874 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:59:07.827236 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:59:07.827240 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:59:07.827742 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:59:07.827922 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:07.827931 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:07.828892 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:07.828898 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:07.829777 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:07.829788 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:07.830609 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:07.830615 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:07.830783 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:07.831405 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:59:07.831423 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:59:07.833394 (+ 1971us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:59:07.836382 (+ 2988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:59:07.836388 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:59:07.836392 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:59:07.836771 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:59:07.837143 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:59:07.837149 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:59:07.837153 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:59:07.837292 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:59:07.837833 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:59:07.837837 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:59:07.838004 (+ 167us) client_negotiation.cc:769] Sending connection context
0924 00:59:07.838147 (+ 143us) client_negotiation.cc:240] Negotiation successful
0924 00:59:07.838455 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260924 00:59:07.839787 22001 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.64.188" port: 42459 } has no permanent_uuid. Determining permanent_uuid...
I20260924 00:59:07.840829 22003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:07.840459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:42459 (local address 127.0.0.1:43430)
0924 00:59:07.840617 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:07.840697 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:42459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260924 00:59:07.841715 22003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:07.841390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:42459 (local address 127.0.0.1:43446)
0924 00:59:07.841525 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:07.841619 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:42459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260924 00:59:07.841933 22001 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.64.188:42459: Network error: Client connection negotiation failed: client connection to 127.8.64.188:42459: connect: Connection refused (error 111)
I20260924 00:59:07.893172 22001 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.64.188" port: 42459 } attempt: 1
I20260924 00:59:07.894558 22003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:07.894016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:42459 (local address 127.0.0.1:43458)
0924 00:59:07.894212 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:07.894382 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:42459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260924 00:59:07.895553 22003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:07.895222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:42459 (local address 127.0.0.1:43470)
0924 00:59:07.895377 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:07.895447 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:42459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260924 00:59:07.895766 22001 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.64.188:42459: Network error: Client connection negotiation failed: client connection to 127.8.64.188:42459: connect: Connection refused (error 111)
I20260924 00:59:07.915306 22076 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.64.188" port: 42459 } attempt: 3
I20260924 00:59:07.916447 22078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:07.915946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:42459 (local address 127.0.0.1:43476)
0924 00:59:07.916180 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:07.916281 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:42459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260924 00:59:07.917495 22078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:07.917090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:42459 (local address 127.0.0.1:43482)
0924 00:59:07.917318 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:07.917390 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:42459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
W20260924 00:59:07.917692 22076 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.64.188:42459: Network error: Client connection negotiation failed: client connection to 127.8.64.188:42459: connect: Connection refused (error 111)
I20260924 00:59:07.964015 22001 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.64.188" port: 42459 } attempt: 2
I20260924 00:59:07.965155 22003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:07.964682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:42459 (local address 127.0.0.1:43494)
0924 00:59:07.964908 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:07.964996 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:42459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260924 00:59:07.966130 22003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:07.965796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:42459 (local address 127.0.0.1:43504)
0924 00:59:07.965950 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:07.966024 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:42459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260924 00:59:07.966352 22001 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.64.188:42459: Network error: Client connection negotiation failed: client connection to 127.8.64.188:42459: connect: Connection refused (error 111)
W20260924 00:59:08.044601 22083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:59:08.044953 22083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:59:08.045123 22083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:59:08.052474 22083 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:59:08.052626 22083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:59:08.052745 22083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:59:08.052845 22083 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:59:08.052937 22083 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:59:08.060889 22076 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.64.188" port: 42459 } attempt: 4
I20260924 00:59:08.062053 22078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:08.061504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:42459 (local address 127.0.0.1:43516)
0924 00:59:08.061762 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:08.061867 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:42459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260924 00:59:08.063158 22078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:08.062792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:42459 (local address 127.0.0.1:43528)
0924 00:59:08.062948 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:08.063031 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:42459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260924 00:59:08.063406 22076 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.64.188:42459: Network error: Client connection negotiation failed: client connection to 127.8.64.188:42459: connect: Connection refused (error 111)
I20260924 00:59:08.063757 22083 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:33379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:36169,127.8.64.189:43831,127.8.64.188:42459
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.188.keytab
--principal=kudu/127.8.64.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.188:42459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.64.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:59:08.065061 22083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:59:08.066438 22083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 00:59:08.072556 22001 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.64.188" port: 42459 } attempt: 3
I20260924 00:59:08.073905 22003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:08.073230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:42459 (local address 127.0.0.1:43534)
0924 00:59:08.073525 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:08.073658 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:42459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260924 00:59:08.075047 22003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:08.074709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:42459 (local address 127.0.0.1:43536)
0924 00:59:08.074893 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:08.074956 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:42459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
W20260924 00:59:08.075276 22001 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.64.188:42459: Network error: Client connection negotiation failed: client connection to 127.8.64.188:42459: connect: Connection refused (error 111)
W20260924 00:59:08.078948 22088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:59:08.078902 22091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:59:08.079396 22089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:59:08.080071 22083 server_base.cc:1034] running on GCE node
I20260924 00:59:08.080829 22083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:59:08.082188 22083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:59:08.083427 22083 hybrid_clock.cc:648] HybridClock initialized: now 1790211548083379 us; error 43 us; skew 500 ppm
I20260924 00:59:08.083777 22083 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:59:08 dist-test-slave-mmhs krb5kdc[21920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211548, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:59:08.086984 22083 init.cc:377] Logged in from keytab as kudu/127.8.64.188@KRBTEST.COM (short username kudu)
I20260924 00:59:08.088388 22083 webserver.cc:460] Webserver started at http://127.8.64.188:43755/ using document root <none> and password file <none>
I20260924 00:59:08.089035 22083 fs_manager.cc:359] Metadata directory not provided
I20260924 00:59:08.089143 22083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:59:08.089383 22083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:59:08.092917 22083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-2/data/instance:
uuid: "3caddaa1ed764b36818b3282ffe74618"
format_stamp: "Formatted at 2026-09-24 00:59:08 on dist-test-slave-mmhs"
server_key: "214da222b84d9e18aedb8e68d19ac3fb"
server_key_iv: "2d3d64c0773fcb21b7630084eafdcd02"
server_key_version: "encryptionkey@0"
I20260924 00:59:08.093912 22083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-2/wal/instance:
uuid: "3caddaa1ed764b36818b3282ffe74618"
format_stamp: "Formatted at 2026-09-24 00:59:08 on dist-test-slave-mmhs"
server_key: "214da222b84d9e18aedb8e68d19ac3fb"
server_key_iv: "2d3d64c0773fcb21b7630084eafdcd02"
server_key_version: "encryptionkey@0"
I20260924 00:59:08.099048 22083 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260924 00:59:08.102437 22098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:08.103721 22083 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260924 00:59:08.103899 22083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-2/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-2/wal
uuid: "3caddaa1ed764b36818b3282ffe74618"
format_stamp: "Formatted at 2026-09-24 00:59:08 on dist-test-slave-mmhs"
server_key: "214da222b84d9e18aedb8e68d19ac3fb"
server_key_iv: "2d3d64c0773fcb21b7630084eafdcd02"
server_key_version: "encryptionkey@0"
I20260924 00:59:08.104090 22083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:59:08.121390 22083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:59:08.125725 22083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:59:08.126016 22083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:59:08.144841 22083 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.188:42459
I20260924 00:59:08.144887 22149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.188:42459 every 8 connection(s)
I20260924 00:59:08.146258 22083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-2/data/info.pb
I20260924 00:59:08.150442 22150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:59:08.153823 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 22083
I20260924 00:59:08.154012 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790211343099518-8450-0/minicluster-data/master-2/wal/instance
I20260924 00:59:08.154640 8450 external_mini_cluster.cc:1428] Setting key 0b6788089267b43284f1a442fbb0e9d1
I20260924 00:59:08.158222 22150 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.64.190" port: 36169 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 00:59:08 dist-test-slave-mmhs krb5kdc[21920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211546, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.188@KRBTEST.COM
Sep 24 00:59:08 dist-test-slave-mmhs krb5kdc[21920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211548, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.188@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:59:08.176083 22004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:08.161998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54408 (local address 127.8.64.190:36169)
0924 00:59:08.162179 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:08.162184 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:08.162858 (+ 674us) server_negotiation.cc:407] Connection header received
0924 00:59:08.163062 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:08.163068 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:08.163140 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:08.163254 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:08.164386 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:08.165070 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:08.166491 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:08.166920 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:08.167048 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:08.170543 (+ 3495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:08.170549 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:08.170552 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:08.170583 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:08.173076 (+ 2493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:08.173744 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:08.173751 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:08.173760 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:08.173890 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:08.174342 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:08.174354 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:08.174357 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:08.174610 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:08.174796 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:08.175385 (+ 589us) server_negotiation.cc:299] Negotiation successful
0924 00:59:08.175732 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260924 00:59:08.177100 22154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:08.161919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:36169 (local address 127.0.0.1:54408)
0924 00:59:08.162715 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:08.162757 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 00:59:08.162934 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:59:08.163308 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:59:08.163315 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:59:08.163964 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:59:08.164233 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:08.164241 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:08.165230 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:08.165235 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:08.166358 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:08.166370 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:08.167150 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:08.167158 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:08.167316 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:08.167919 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:59:08.167940 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:59:08.170347 (+ 2407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:59:08.173207 (+ 2860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:59:08.173220 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:59:08.173232 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:59:08.173618 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:59:08.174003 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:59:08.174008 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:59:08.174012 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:59:08.174173 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:59:08.174798 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:59:08.174808 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:59:08.175006 (+ 198us) client_negotiation.cc:769] Sending connection context
0924 00:59:08.175252 (+ 246us) client_negotiation.cc:240] Negotiation successful
0924 00:59:08.175767 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":443,"threads_started":1}
I20260924 00:59:08.177187 22153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:08.156571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43540 (local address 127.8.64.188:42459)
0924 00:59:08.157359 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:08.157371 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:08.157410 (+ 39us) server_negotiation.cc:407] Connection header received
0924 00:59:08.159072 (+ 1662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:08.159094 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:08.159499 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:08.159838 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:08.161732 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:08.163543 (+ 1811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:08.164958 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:08.165649 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:08.165852 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:08.170214 (+ 4362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:08.170222 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:08.170234 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:08.170268 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:08.173123 (+ 2855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:08.173833 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:08.173840 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:08.173842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:08.173991 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:08.174406 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:08.174413 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:08.174417 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:08.174976 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:08.175295 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:08.176299 (+ 1004us) server_negotiation.cc:299] Negotiation successful
0924 00:59:08.176505 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":372,"threads_started":1}
I20260924 00:59:08.178937 22150 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43831 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 00:59:08 dist-test-slave-mmhs krb5kdc[21920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211548, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.188@KRBTEST.COM for kudu/127.8.64.189@KRBTEST.COM
Sep 24 00:59:08 dist-test-slave-mmhs krb5kdc[21920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211546, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:59:08.193472 22080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:08.179713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49558 (local address 127.8.64.189:43831)
0924 00:59:08.179911 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:08.179919 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:08.180013 (+ 94us) server_negotiation.cc:407] Connection header received
0924 00:59:08.180158 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:08.180163 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:08.180232 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:08.180341 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:08.181209 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:08.181875 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:08.183217 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:08.183651 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:08.183743 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:08.187210 (+ 3467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:08.187218 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:08.187225 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:08.187273 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:08.190578 (+ 3305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:08.191413 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:08.191421 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:08.191428 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:08.191568 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:08.191994 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:08.191997 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:08.191999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:08.192376 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:08.192512 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:08.193036 (+ 524us) server_negotiation.cc:299] Negotiation successful
0924 00:59:08.193198 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260924 00:59:08.193648 22154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:08.179754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:43831 (local address 127.0.0.1:49558)
0924 00:59:08.179924 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:08.179944 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0924 00:59:08.180054 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:59:08.180360 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:59:08.180364 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:59:08.180837 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:59:08.181056 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:08.181068 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:08.182012 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:08.182017 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:08.183072 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:08.183084 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:08.183808 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:08.183814 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:08.184010 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:08.184572 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:59:08.184598 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:59:08.187009 (+ 2411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:59:08.190721 (+ 3712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:59:08.190726 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:59:08.190730 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:59:08.191261 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:59:08.191694 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:59:08.191698 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:59:08.191702 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:59:08.191881 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:59:08.192553 (+ 672us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:59:08.192560 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:59:08.192770 (+ 210us) client_negotiation.cc:769] Sending connection context
0924 00:59:08.192950 (+ 180us) client_negotiation.cc:240] Negotiation successful
0924 00:59:08.193250 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260924 00:59:08.194926 22150 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.64.188" port: 42459 } has no permanent_uuid. Determining permanent_uuid...
I20260924 00:59:08.197237 22004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:08.180510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54422 (local address 127.8.64.190:36169)
0924 00:59:08.180665 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:08.180670 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:08.180787 (+ 117us) server_negotiation.cc:407] Connection header received
0924 00:59:08.180931 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:08.180934 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:08.180986 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:08.181069 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:08.182388 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:08.183116 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:08.184871 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:08.185277 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:08.185374 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:08.190518 (+ 5144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:08.190524 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:08.190527 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:08.190561 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:08.194007 (+ 3446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:08.194970 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:08.194975 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:08.194978 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:08.195143 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:08.195631 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:08.195640 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:08.195644 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:08.196014 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:08.196310 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:08.196666 (+ 356us) server_negotiation.cc:299] Negotiation successful
0924 00:59:08.196890 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
Sep 24 00:59:08 dist-test-slave-mmhs krb5kdc[21920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211548, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.188@KRBTEST.COM for kudu/127.8.64.188@KRBTEST.COM
Sep 24 00:59:08 dist-test-slave-mmhs krb5kdc[21920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211546, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.189@KRBTEST.COM
I20260924 00:59:08.208350 22154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:08.195680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:42459 (local address 127.0.0.1:43554)
0924 00:59:08.195839 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:08.195857 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0924 00:59:08.195957 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:59:08.196351 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:59:08.196355 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:59:08.196807 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:59:08.196976 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:08.197000 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:08.198152 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:08.198160 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:08.199241 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:08.199252 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:08.200029 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:08.200037 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:08.200250 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:08.200858 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:59:08.200882 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:59:08.202981 (+ 2099us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:59:08.205909 (+ 2928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:59:08.205914 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:59:08.205918 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:59:08.206254 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:59:08.206646 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:59:08.206650 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:59:08.206654 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:59:08.206784 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:59:08.207322 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:59:08.207326 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:59:08.207464 (+ 138us) client_negotiation.cc:769] Sending connection context
0924 00:59:08.207619 (+ 155us) client_negotiation.cc:240] Negotiation successful
0924 00:59:08.208052 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260924 00:59:08.208696 22153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:08.195779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43554 (local address 127.8.64.188:42459)
0924 00:59:08.195914 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:08.195918 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:08.195932 (+ 14us) server_negotiation.cc:407] Connection header received
0924 00:59:08.196108 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:08.196113 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:08.196180 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:08.196297 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:08.197113 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:08.197993 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:08.199345 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:08.199875 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:08.199969 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:08.203156 (+ 3187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:08.203162 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:08.203166 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:08.203202 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:08.205777 (+ 2575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:08.206417 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:08.206422 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:08.206426 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:08.206534 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:08.206911 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:08.206917 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:08.206921 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:08.207214 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:08.207358 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:08.208055 (+ 697us) server_negotiation.cc:299] Negotiation successful
0924 00:59:08.208306 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260924 00:59:08.213343 22150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3caddaa1ed764b36818b3282ffe74618: Bootstrap starting.
I20260924 00:59:08.213505 22080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:08.199056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49568 (local address 127.8.64.189:43831)
0924 00:59:08.199235 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:08.199242 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:08.199260 (+ 18us) server_negotiation.cc:407] Connection header received
0924 00:59:08.199311 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:08.199316 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:08.199384 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:08.199477 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:08.201029 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:08.201734 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:08.203150 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:08.203595 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:08.203735 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:08.207959 (+ 4224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:08.207966 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:08.207970 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:08.208017 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:08.210905 (+ 2888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:08.211579 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:08.211586 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:08.211589 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:08.211734 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:08.212110 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:08.212114 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:08.212118 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:08.212378 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:08.212677 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:08.212943 (+ 266us) server_negotiation.cc:299] Negotiation successful
0924 00:59:08.213175 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260924 00:59:08.216910 22150 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3caddaa1ed764b36818b3282ffe74618: Neither blocks nor log segments found. Creating new log.
I20260924 00:59:08.218230 22150 log.cc:824] T 00000000000000000000000000000000 P 3caddaa1ed764b36818b3282ffe74618: Log is configured to *not* fsync() on all Append() calls
I20260924 00:59:08.219035 8450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 00:59:08.220849 22150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3caddaa1ed764b36818b3282ffe74618: No bootstrap required, opened a new log
I20260924 00:59:08.233554 22150 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3caddaa1ed764b36818b3282ffe74618 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3580661dd79443dfb823fc61af395a90" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 36169 } } peers { permanent_uuid: "6958b0521072447f8df5b1db8a3ef826" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43831 } } peers { permanent_uuid: "3caddaa1ed764b36818b3282ffe74618" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 42459 } }
I20260924 00:59:08.234182 22150 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3caddaa1ed764b36818b3282ffe74618 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:59:08.234401 22150 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3caddaa1ed764b36818b3282ffe74618 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3caddaa1ed764b36818b3282ffe74618, State: Initialized, Role: FOLLOWER
I20260924 00:59:08.235260 22150 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3caddaa1ed764b36818b3282ffe74618 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3580661dd79443dfb823fc61af395a90" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 36169 } } peers { permanent_uuid: "6958b0521072447f8df5b1db8a3ef826" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43831 } } peers { permanent_uuid: "3caddaa1ed764b36818b3282ffe74618" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 42459 } }
I20260924 00:59:08.241397 22004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:08.224433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54430 (local address 127.8.64.190:36169)
0924 00:59:08.224623 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:08.224629 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:08.225107 (+ 478us) server_negotiation.cc:407] Connection header received
0924 00:59:08.225309 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:08.225315 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:08.225393 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:08.225499 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:08.227307 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:08.228028 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:08.229404 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:08.229834 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:08.229966 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:08.234167 (+ 4201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:08.234176 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:08.234183 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:08.234227 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:08.237749 (+ 3522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:08.238672 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:08.238680 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:08.238684 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:08.238858 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:08.239374 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:08.239380 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:08.239384 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:08.239765 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:08.240009 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:08.240723 (+ 714us) server_negotiation.cc:299] Negotiation successful
0924 00:59:08.240991 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260924 00:59:08.242380 22163 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3caddaa1ed764b36818b3282ffe74618 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3580661dd79443dfb823fc61af395a90" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 36169 } } peers { permanent_uuid: "6958b0521072447f8df5b1db8a3ef826" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43831 } } peers { permanent_uuid: "3caddaa1ed764b36818b3282ffe74618" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 42459 } } }
I20260924 00:59:08.244638 22150 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3caddaa1ed764b36818b3282ffe74618 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:59:08.244818 22163 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3caddaa1ed764b36818b3282ffe74618 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:59:08.249226 22080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:08.224348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49584 (local address 127.8.64.189:43831)
0924 00:59:08.224537 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:08.224546 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:08.230690 (+ 6144us) server_negotiation.cc:407] Connection header received
0924 00:59:08.230968 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:08.230973 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:08.231060 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:08.231136 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:08.234235 (+ 3099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:08.235399 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:08.237556 (+ 2157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:08.238235 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:08.238358 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:08.241945 (+ 3587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:08.241951 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:08.241955 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:08.242001 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:08.245682 (+ 3681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:08.246597 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:08.246603 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:08.246607 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:08.246809 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:08.247253 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:08.247260 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:08.247263 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:08.247635 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:08.247889 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:08.248537 (+ 648us) server_negotiation.cc:299] Negotiation successful
0924 00:59:08.248865 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260924 00:59:08.251425 22153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:08.230843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43566 (local address 127.8.64.188:42459)
0924 00:59:08.231059 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:08.231067 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:08.232271 (+ 1204us) server_negotiation.cc:407] Connection header received
0924 00:59:08.232461 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:08.232466 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:08.232553 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:08.232647 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:08.234676 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:08.235745 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:08.238460 (+ 2715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:08.239443 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:08.239572 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:08.244060 (+ 4488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:08.244070 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:08.244076 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:08.244121 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:08.248004 (+ 3883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:08.248953 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:08.248959 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:08.248963 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:08.249121 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:08.249586 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:08.249593 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:08.249597 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:08.249971 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:08.250257 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:08.250702 (+ 445us) server_negotiation.cc:299] Negotiation successful
0924 00:59:08.250980 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260924 00:59:08.252758 22001 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.64.188" port: 42459 } attempt: 4
W20260924 00:59:08.259553 22174 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3caddaa1ed764b36818b3282ffe74618: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 00:59:08.259791 22174 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Sep 24 00:59:08 dist-test-slave-mmhs krb5kdc[21920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211547, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for kudu/127.8.64.188@KRBTEST.COM
I20260924 00:59:08.254943 22083 master_runner.cc:186] Error getting master registration for 127.8.64.190:36169: OK, instance_id { permanent_uuid: "3580661dd79443dfb823fc61af395a90" instance_seqno: 1790211547115706 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 00:59:08.267550 22083 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:59:08.270219 22153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:08.253359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43576 (local address 127.8.64.188:42459)
0924 00:59:08.253800 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:08.253803 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:08.253815 (+ 12us) server_negotiation.cc:407] Connection header received
0924 00:59:08.253849 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:08.253852 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:08.253899 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:08.253999 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:08.254922 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:08.256202 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:08.260064 (+ 3862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:08.260627 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:08.260776 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:08.264756 (+ 3980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:08.264762 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:08.264767 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:08.264807 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:08.267624 (+ 2817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:08.268220 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:08.268226 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:08.268229 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:08.268363 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:08.268760 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:08.268765 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:08.268768 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:08.269087 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:08.269246 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:08.269649 (+ 403us) server_negotiation.cc:299] Negotiation successful
0924 00:59:08.269857 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359}
I20260924 00:59:08.270242 22003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:08.253354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:42459 (local address 127.0.0.1:43576)
0924 00:59:08.253541 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:08.253567 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0924 00:59:08.253653 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:59:08.254046 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:59:08.254051 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:59:08.254531 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:59:08.254759 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:08.254772 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:08.258433 (+ 3661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:08.258439 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:08.259921 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:08.259940 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:08.260792 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:08.260798 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:08.260984 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:08.261626 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:59:08.261651 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:59:08.264593 (+ 2942us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:59:08.267740 (+ 3147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:59:08.267746 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:59:08.267750 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:59:08.268090 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:59:08.268477 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:59:08.268485 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:59:08.268488 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:59:08.268628 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:59:08.269211 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:59:08.269216 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:59:08.269388 (+ 172us) client_negotiation.cc:769] Sending connection context
0924 00:59:08.269520 (+ 132us) client_negotiation.cc:240] Negotiation successful
0924 00:59:08.269835 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260924 00:59:08.276189 22001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3580661dd79443dfb823fc61af395a90: Bootstrap starting.
I20260924 00:59:08.280154 22001 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3580661dd79443dfb823fc61af395a90: Neither blocks nor log segments found. Creating new log.
I20260924 00:59:08.281409 22001 log.cc:824] T 00000000000000000000000000000000 P 3580661dd79443dfb823fc61af395a90: Log is configured to *not* fsync() on all Append() calls
I20260924 00:59:08.284361 22001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3580661dd79443dfb823fc61af395a90: No bootstrap required, opened a new log
I20260924 00:59:08.292706 22001 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3580661dd79443dfb823fc61af395a90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3580661dd79443dfb823fc61af395a90" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 36169 } } peers { permanent_uuid: "6958b0521072447f8df5b1db8a3ef826" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43831 } } peers { permanent_uuid: "3caddaa1ed764b36818b3282ffe74618" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 42459 } }
I20260924 00:59:08.293016 22001 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3580661dd79443dfb823fc61af395a90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:59:08.293192 22001 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3580661dd79443dfb823fc61af395a90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3580661dd79443dfb823fc61af395a90, State: Initialized, Role: FOLLOWER
I20260924 00:59:08.293823 22001 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3580661dd79443dfb823fc61af395a90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3580661dd79443dfb823fc61af395a90" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 36169 } } peers { permanent_uuid: "6958b0521072447f8df5b1db8a3ef826" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43831 } } peers { permanent_uuid: "3caddaa1ed764b36818b3282ffe74618" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 42459 } }
I20260924 00:59:08.294817 22177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3580661dd79443dfb823fc61af395a90 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3580661dd79443dfb823fc61af395a90" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 36169 } } peers { permanent_uuid: "6958b0521072447f8df5b1db8a3ef826" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43831 } } peers { permanent_uuid: "3caddaa1ed764b36818b3282ffe74618" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 42459 } } }
I20260924 00:59:08.295269 22177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3580661dd79443dfb823fc61af395a90 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:59:08.295902 22001 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3580661dd79443dfb823fc61af395a90 [sys.catalog]: configured and running, proceeding with master startup.
W20260924 00:59:08.307394 22188 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3580661dd79443dfb823fc61af395a90: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 00:59:08.307808 22188 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 00:59:08.306653 21934 master_runner.cc:186] Error getting master registration for 127.8.64.189:43831: OK, instance_id { permanent_uuid: "6958b0521072447f8df5b1db8a3ef826" instance_seqno: 1790211547636407 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 00:59:08.318382 21934 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260924 00:59:08.355711 21955 tablet.cc:2369] T 00000000000000000000000000000000 P 3580661dd79443dfb823fc61af395a90: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:59:08.362632 22076 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.64.188" port: 42459 } attempt: 5
Sep 24 00:59:08 dist-test-slave-mmhs krb5kdc[21920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211547, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.189@KRBTEST.COM for kudu/127.8.64.188@KRBTEST.COM
W20260924 00:59:08.378168 22104 tablet.cc:2369] T 00000000000000000000000000000000 P 3caddaa1ed764b36818b3282ffe74618: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:59:08.382061 22078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:08.363363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:42459 (local address 127.0.0.1:43586)
0924 00:59:08.363643 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:08.363664 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0924 00:59:08.363792 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:59:08.364186 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:59:08.364192 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:59:08.364711 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:59:08.364915 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:08.364927 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:08.366477 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:08.366483 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:08.367930 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:08.367945 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:08.369915 (+ 1970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:08.369922 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:08.370128 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:08.370845 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:59:08.370869 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:59:08.373690 (+ 2821us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:59:08.379245 (+ 5555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:59:08.379251 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:59:08.379255 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:59:08.379791 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:59:08.380231 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:59:08.380236 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:59:08.380240 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:59:08.380406 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:59:08.381025 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:59:08.381030 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:59:08.381194 (+ 164us) client_negotiation.cc:769] Sending connection context
0924 00:59:08.381365 (+ 171us) client_negotiation.cc:240] Negotiation successful
0924 00:59:08.381662 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153}
I20260924 00:59:08.382117 22153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:08.363461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43586 (local address 127.8.64.188:42459)
0924 00:59:08.363648 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:08.363654 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:08.363745 (+ 91us) server_negotiation.cc:407] Connection header received
0924 00:59:08.363920 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:08.363926 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:08.364017 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:08.364134 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:08.365079 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:08.366267 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:08.369036 (+ 2769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:08.369756 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:08.369887 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:08.373909 (+ 4022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:08.373916 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:08.373921 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:08.373960 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:08.377799 (+ 3839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:08.379925 (+ 2126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:08.379936 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:08.379940 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:08.380087 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:08.380521 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:08.380526 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:08.380531 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:08.380899 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:08.381047 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:08.381496 (+ 449us) server_negotiation.cc:299] Negotiation successful
0924 00:59:08.381734 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260924 00:59:08.389514 22076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6958b0521072447f8df5b1db8a3ef826: Bootstrap starting.
I20260924 00:59:08.393620 22076 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6958b0521072447f8df5b1db8a3ef826: Neither blocks nor log segments found. Creating new log.
I20260924 00:59:08.394968 22076 log.cc:824] T 00000000000000000000000000000000 P 6958b0521072447f8df5b1db8a3ef826: Log is configured to *not* fsync() on all Append() calls
I20260924 00:59:08.399679 22076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6958b0521072447f8df5b1db8a3ef826: No bootstrap required, opened a new log
I20260924 00:59:08.410843 22076 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6958b0521072447f8df5b1db8a3ef826 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3580661dd79443dfb823fc61af395a90" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 36169 } } peers { permanent_uuid: "6958b0521072447f8df5b1db8a3ef826" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43831 } } peers { permanent_uuid: "3caddaa1ed764b36818b3282ffe74618" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 42459 } }
I20260924 00:59:08.411273 22076 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6958b0521072447f8df5b1db8a3ef826 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:59:08.411391 22076 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6958b0521072447f8df5b1db8a3ef826 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6958b0521072447f8df5b1db8a3ef826, State: Initialized, Role: FOLLOWER
I20260924 00:59:08.412051 22076 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6958b0521072447f8df5b1db8a3ef826 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3580661dd79443dfb823fc61af395a90" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 36169 } } peers { permanent_uuid: "6958b0521072447f8df5b1db8a3ef826" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43831 } } peers { permanent_uuid: "3caddaa1ed764b36818b3282ffe74618" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 42459 } }
I20260924 00:59:08.413074 22191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6958b0521072447f8df5b1db8a3ef826 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3580661dd79443dfb823fc61af395a90" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 36169 } } peers { permanent_uuid: "6958b0521072447f8df5b1db8a3ef826" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43831 } } peers { permanent_uuid: "3caddaa1ed764b36818b3282ffe74618" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 42459 } } }
I20260924 00:59:08.413573 22191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6958b0521072447f8df5b1db8a3ef826 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:59:08.414412 22076 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6958b0521072447f8df5b1db8a3ef826 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:59:08.433048 22008 master_runner.cc:263] No leader master found from master 6958b0521072447f8df5b1db8a3ef826
I20260924 00:59:08.433214 22008 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260924 00:59:08.439008 22202 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6958b0521072447f8df5b1db8a3ef826: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 00:59:08.439253 22202 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 00:59:08.444149 22177 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3580661dd79443dfb823fc61af395a90 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:59:08.444446 22177 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3580661dd79443dfb823fc61af395a90 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3580661dd79443dfb823fc61af395a90" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 36169 } } peers { permanent_uuid: "6958b0521072447f8df5b1db8a3ef826" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43831 } } peers { permanent_uuid: "3caddaa1ed764b36818b3282ffe74618" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 42459 } }
I20260924 00:59:08.446030 22177 leader_election.cc:288] T 00000000000000000000000000000000 P 3580661dd79443dfb823fc61af395a90 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6958b0521072447f8df5b1db8a3ef826 (127.8.64.189:43831), 3caddaa1ed764b36818b3282ffe74618 (127.8.64.188:42459)
I20260924 00:59:08.447412 22050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3580661dd79443dfb823fc61af395a90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6958b0521072447f8df5b1db8a3ef826" is_pre_election: true
I20260924 00:59:08.448009 22050 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6958b0521072447f8df5b1db8a3ef826 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3580661dd79443dfb823fc61af395a90 in term 0.
I20260924 00:59:08.448647 22125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3580661dd79443dfb823fc61af395a90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3caddaa1ed764b36818b3282ffe74618" is_pre_election: true
I20260924 00:59:08.449199 22125 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3caddaa1ed764b36818b3282ffe74618 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3580661dd79443dfb823fc61af395a90 in term 0.
I20260924 00:59:08.449982 21950 leader_election.cc:302] T 00000000000000000000000000000000 P 3580661dd79443dfb823fc61af395a90 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3580661dd79443dfb823fc61af395a90, 3caddaa1ed764b36818b3282ffe74618; no voters:
I20260924 00:59:08.450947 22177 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3580661dd79443dfb823fc61af395a90 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:59:08.451116 22177 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3580661dd79443dfb823fc61af395a90 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:59:08.451223 22177 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3580661dd79443dfb823fc61af395a90 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:59:08.454646 22177 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3580661dd79443dfb823fc61af395a90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3580661dd79443dfb823fc61af395a90" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 36169 } } peers { permanent_uuid: "6958b0521072447f8df5b1db8a3ef826" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43831 } } peers { permanent_uuid: "3caddaa1ed764b36818b3282ffe74618" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 42459 } }
I20260924 00:59:08.457214 22050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3580661dd79443dfb823fc61af395a90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6958b0521072447f8df5b1db8a3ef826"
I20260924 00:59:08.457425 22050 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6958b0521072447f8df5b1db8a3ef826 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:59:08.460952 22050 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6958b0521072447f8df5b1db8a3ef826 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3580661dd79443dfb823fc61af395a90 in term 1.
I20260924 00:59:08.461445 21951 leader_election.cc:302] T 00000000000000000000000000000000 P 3580661dd79443dfb823fc61af395a90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3580661dd79443dfb823fc61af395a90, 6958b0521072447f8df5b1db8a3ef826; no voters:
I20260924 00:59:08.465029 22125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3580661dd79443dfb823fc61af395a90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3caddaa1ed764b36818b3282ffe74618"
I20260924 00:59:08.465267 22125 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3caddaa1ed764b36818b3282ffe74618 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:59:08.468868 22125 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3caddaa1ed764b36818b3282ffe74618 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3580661dd79443dfb823fc61af395a90 in term 1.
I20260924 00:59:08.469795 22177 leader_election.cc:288] T 00000000000000000000000000000000 P 3580661dd79443dfb823fc61af395a90 [CANDIDATE]: Term 1 election: Requested vote from peers 6958b0521072447f8df5b1db8a3ef826 (127.8.64.189:43831), 3caddaa1ed764b36818b3282ffe74618 (127.8.64.188:42459)
I20260924 00:59:08.470207 22177 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3580661dd79443dfb823fc61af395a90 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:59:08.470687 22177 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3580661dd79443dfb823fc61af395a90 [term 1 LEADER]: Becoming Leader. State: Replica: 3580661dd79443dfb823fc61af395a90, State: Running, Role: LEADER
I20260924 00:59:08.471565 22177 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3580661dd79443dfb823fc61af395a90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3580661dd79443dfb823fc61af395a90" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 36169 } } peers { permanent_uuid: "6958b0521072447f8df5b1db8a3ef826" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43831 } } peers { permanent_uuid: "3caddaa1ed764b36818b3282ffe74618" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 42459 } }
I20260924 00:59:08.488202 22204 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3580661dd79443dfb823fc61af395a90 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3580661dd79443dfb823fc61af395a90. Latest consensus state: current_term: 1 leader_uuid: "3580661dd79443dfb823fc61af395a90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3580661dd79443dfb823fc61af395a90" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 36169 } } peers { permanent_uuid: "6958b0521072447f8df5b1db8a3ef826" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43831 } } peers { permanent_uuid: "3caddaa1ed764b36818b3282ffe74618" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 42459 } } }
I20260924 00:59:08.488812 22204 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3580661dd79443dfb823fc61af395a90 [sys.catalog]: This master's current role is: LEADER
I20260924 00:59:08.495759 22207 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:59:08.498273 22207 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:59:08.516739 22125 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3caddaa1ed764b36818b3282ffe74618 [term 1 FOLLOWER]: Refusing update from remote peer 3580661dd79443dfb823fc61af395a90: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:59:08.517925 22204 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3580661dd79443dfb823fc61af395a90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3caddaa1ed764b36818b3282ffe74618" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 42459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:59:08.525602 22050 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6958b0521072447f8df5b1db8a3ef826 [term 1 FOLLOWER]: Refusing update from remote peer 3580661dd79443dfb823fc61af395a90: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:59:08.526768 22204 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3580661dd79443dfb823fc61af395a90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6958b0521072447f8df5b1db8a3ef826" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:59:08.563449 22191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6958b0521072447f8df5b1db8a3ef826 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3580661dd79443dfb823fc61af395a90. Latest consensus state: current_term: 1 leader_uuid: "3580661dd79443dfb823fc61af395a90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3580661dd79443dfb823fc61af395a90" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 36169 } } peers { permanent_uuid: "6958b0521072447f8df5b1db8a3ef826" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43831 } } peers { permanent_uuid: "3caddaa1ed764b36818b3282ffe74618" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 42459 } } }
I20260924 00:59:08.563841 22191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6958b0521072447f8df5b1db8a3ef826 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:59:08.569725 22208 mvcc.cc:205] Tried to move back new op lower bound from 7332706502711123968 to 7332706502552141824. Current Snapshot: MvccSnapshot[applied={T|T < 7332706502711123968}]
I20260924 00:59:08.573421 22210 mvcc.cc:205] Tried to move back new op lower bound from 7332706502711123968 to 7332706502552141824. Current Snapshot: MvccSnapshot[applied={T|T < 7332706502711123968}]
I20260924 00:59:08.580971 22163 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3caddaa1ed764b36818b3282ffe74618 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3580661dd79443dfb823fc61af395a90. Latest consensus state: current_term: 1 leader_uuid: "3580661dd79443dfb823fc61af395a90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3580661dd79443dfb823fc61af395a90" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 36169 } } peers { permanent_uuid: "6958b0521072447f8df5b1db8a3ef826" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43831 } } peers { permanent_uuid: "3caddaa1ed764b36818b3282ffe74618" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 42459 } } }
I20260924 00:59:08.581334 22163 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3caddaa1ed764b36818b3282ffe74618 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:59:08.583704 22207 catalog_manager.cc:1347] Generated new cluster ID: 80cffd3d83614a3cbdfa8d6c9a43bb9e
I20260924 00:59:08.583844 22207 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:59:08.589462 22191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6958b0521072447f8df5b1db8a3ef826 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3580661dd79443dfb823fc61af395a90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3580661dd79443dfb823fc61af395a90" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 36169 } } peers { permanent_uuid: "6958b0521072447f8df5b1db8a3ef826" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43831 } } peers { permanent_uuid: "3caddaa1ed764b36818b3282ffe74618" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 42459 } } }
I20260924 00:59:08.589718 22191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6958b0521072447f8df5b1db8a3ef826 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:59:08.598242 22177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3580661dd79443dfb823fc61af395a90 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3580661dd79443dfb823fc61af395a90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3580661dd79443dfb823fc61af395a90" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 36169 } } peers { permanent_uuid: "6958b0521072447f8df5b1db8a3ef826" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43831 } } peers { permanent_uuid: "3caddaa1ed764b36818b3282ffe74618" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 42459 } } }
I20260924 00:59:08.598639 22177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3580661dd79443dfb823fc61af395a90 [sys.catalog]: This master's current role is: LEADER
I20260924 00:59:08.608913 22204 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3580661dd79443dfb823fc61af395a90 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3580661dd79443dfb823fc61af395a90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3580661dd79443dfb823fc61af395a90" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 36169 } } peers { permanent_uuid: "6958b0521072447f8df5b1db8a3ef826" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43831 } } peers { permanent_uuid: "3caddaa1ed764b36818b3282ffe74618" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 42459 } } }
I20260924 00:59:08.609750 22204 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3580661dd79443dfb823fc61af395a90 [sys.catalog]: This master's current role is: LEADER
I20260924 00:59:08.611249 22209 mvcc.cc:205] Tried to move back new op lower bound from 7332706502711123968 to 7332706502552141824. Current Snapshot: MvccSnapshot[applied={T|T < 7332706502711123968}]
I20260924 00:59:08.620656 22163 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3caddaa1ed764b36818b3282ffe74618 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3580661dd79443dfb823fc61af395a90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3580661dd79443dfb823fc61af395a90" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 36169 } } peers { permanent_uuid: "6958b0521072447f8df5b1db8a3ef826" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43831 } } peers { permanent_uuid: "3caddaa1ed764b36818b3282ffe74618" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 42459 } } }
I20260924 00:59:08.620961 22163 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3caddaa1ed764b36818b3282ffe74618 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:59:08.643642 22207 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:59:08.645526 22207 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:59:08.668442 22207 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3580661dd79443dfb823fc61af395a90: Generated new TSK 0
I20260924 00:59:08.669636 22207 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 00:59:09.261785 22174 catalog_manager.cc:1259] Loaded cluster ID: 80cffd3d83614a3cbdfa8d6c9a43bb9e
I20260924 00:59:09.261888 22174 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3caddaa1ed764b36818b3282ffe74618: loading cluster ID for follower catalog manager: success
I20260924 00:59:09.264331 22174 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3caddaa1ed764b36818b3282ffe74618: acquiring CA information for follower catalog manager: success
I20260924 00:59:09.266117 22174 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3caddaa1ed764b36818b3282ffe74618: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 00:59:09.441256 22202 catalog_manager.cc:1259] Loaded cluster ID: 80cffd3d83614a3cbdfa8d6c9a43bb9e
I20260924 00:59:09.441366 22202 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6958b0521072447f8df5b1db8a3ef826: loading cluster ID for follower catalog manager: success
I20260924 00:59:09.443841 22202 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6958b0521072447f8df5b1db8a3ef826: acquiring CA information for follower catalog manager: success
I20260924 00:59:09.445631 22202 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6958b0521072447f8df5b1db8a3ef826: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 00:59:09.812805 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 21934
I20260924 00:59:09.813918 21996 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:59:09.925542 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 21934
I20260924 00:59:09.947086 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 22008
I20260924 00:59:09.948289 22071 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:59:10.053478 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 22008
I20260924 00:59:10.074011 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 22083
I20260924 00:59:10.075428 22145 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:59:10.183748 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 22083
2026-09-24T00:59:10Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4846 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790211343099518-8450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:59:10 dist-test-slave-mmhs krb5kdc[22225](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:59:10 dist-test-slave-mmhs krb5kdc[22225](info): set up 2 sockets
Sep 24 00:59:10 dist-test-slave-mmhs krb5kdc[22225](info): commencing operation
krb5kdc: starting...
W20260924 00:59:11.296471 8450 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Sep 24 00:59:11 dist-test-slave-mmhs krb5kdc[22225](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:59:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:59:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.64.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
I20260924 00:59:11.536204 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:45719
--webserver_interface=127.8.64.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:44419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:45719
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:59:11.862744 22238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:59:11.863029 22238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:59:11.863117 22238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:59:11.870268 22238 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:59:11.870393 22238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:59:11.870440 22238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:59:11.870496 22238 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:59:11.870541 22238 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:59:11.880255 22238 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:44419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:45719
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:45719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:59:11.881515 22238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:59:11.882867 22238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:59:11.895171 22243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:59:11.895151 22246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:59:11.895105 22244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:59:11.896240 22238 server_base.cc:1034] running on GCE node
I20260924 00:59:11.897089 22238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:59:11.898619 22238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:59:11.899855 22238 hybrid_clock.cc:648] HybridClock initialized: now 1790211551899810 us; error 40 us; skew 500 ppm
I20260924 00:59:11.900213 22238 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:59:11 dist-test-slave-mmhs krb5kdc[22225](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211551, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:59:11.903308 22238 init.cc:377] Logged in from keytab as kudu/127.8.64.190@KRBTEST.COM (short username kudu)
I20260924 00:59:11.904850 22238 webserver.cc:460] Webserver started at http://127.8.64.190:46771/ using document root <none> and password file <none>
I20260924 00:59:11.905534 22238 fs_manager.cc:359] Metadata directory not provided
I20260924 00:59:11.905682 22238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:59:11.906004 22238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:59:11.908991 22238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "5e6eedd88ce94e379a63aa9adb7c6e74"
format_stamp: "Formatted at 2026-09-24 00:59:11 on dist-test-slave-mmhs"
server_key: "b34d0d3ee85bf307c45465f95f376aac"
server_key_iv: "fc0ee96d3afaa270d607e4f1546c269e"
server_key_version: "encryptionkey@0"
I20260924 00:59:11.909744 22238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "5e6eedd88ce94e379a63aa9adb7c6e74"
format_stamp: "Formatted at 2026-09-24 00:59:11 on dist-test-slave-mmhs"
server_key: "b34d0d3ee85bf307c45465f95f376aac"
server_key_iv: "fc0ee96d3afaa270d607e4f1546c269e"
server_key_version: "encryptionkey@0"
I20260924 00:59:11.914880 22238 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260924 00:59:11.918525 22253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:11.919831 22238 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260924 00:59:11.920024 22238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "5e6eedd88ce94e379a63aa9adb7c6e74"
format_stamp: "Formatted at 2026-09-24 00:59:11 on dist-test-slave-mmhs"
server_key: "b34d0d3ee85bf307c45465f95f376aac"
server_key_iv: "fc0ee96d3afaa270d607e4f1546c269e"
server_key_version: "encryptionkey@0"
I20260924 00:59:11.920223 22238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:59:11.946727 22238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:59:11.951141 22238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:59:11.951439 22238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:59:11.970284 22238 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.190:45719
I20260924 00:59:11.970368 22304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.190:45719 every 8 connection(s)
I20260924 00:59:11.971693 22238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 00:59:11.975317 22305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:59:11.979305 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 22238
I20260924 00:59:11.979527 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 00:59:11.980194 8450 external_mini_cluster.cc:1428] Setting key 99672714c271d92dee7e4fd3751d4086
I20260924 00:59:11.986466 22305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e6eedd88ce94e379a63aa9adb7c6e74: Bootstrap starting.
I20260924 00:59:11.990041 22305 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e6eedd88ce94e379a63aa9adb7c6e74: Neither blocks nor log segments found. Creating new log.
I20260924 00:59:11.991217 22305 log.cc:824] T 00000000000000000000000000000000 P 5e6eedd88ce94e379a63aa9adb7c6e74: Log is configured to *not* fsync() on all Append() calls
I20260924 00:59:11.994787 22305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e6eedd88ce94e379a63aa9adb7c6e74: No bootstrap required, opened a new log
Sep 24 00:59:11 dist-test-slave-mmhs krb5kdc[22225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:59:12.005272 22308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:11.981978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47576 (local address 127.8.64.190:45719)
0924 00:59:11.982772 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:11.982782 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:11.982823 (+ 41us) server_negotiation.cc:407] Connection header received
0924 00:59:11.984986 (+ 2163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:11.985009 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:11.985377 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:11.985691 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:11.987624 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:11.988964 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:11.990818 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:11.991535 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:11.991739 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:11.996616 (+ 4877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:11.996627 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:11.996647 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:11.996696 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:12.000792 (+ 4096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:12.001460 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:12.001468 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:12.001477 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:12.001693 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:12.002063 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:12.002068 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:12.002071 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:12.002765 (+ 694us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:12.003092 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:12.003423 (+ 331us) server_negotiation.cc:299] Negotiation successful
0924 00:59:12.003889 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":387,"threads_started":1}
I20260924 00:59:12.005700 22305 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e6eedd88ce94e379a63aa9adb7c6e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e6eedd88ce94e379a63aa9adb7c6e74" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45719 } }
I20260924 00:59:12.006086 22305 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e6eedd88ce94e379a63aa9adb7c6e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:59:12.006196 22305 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e6eedd88ce94e379a63aa9adb7c6e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e6eedd88ce94e379a63aa9adb7c6e74, State: Initialized, Role: FOLLOWER
I20260924 00:59:12.006889 22305 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e6eedd88ce94e379a63aa9adb7c6e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e6eedd88ce94e379a63aa9adb7c6e74" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45719 } }
I20260924 00:59:12.007148 22305 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e6eedd88ce94e379a63aa9adb7c6e74 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:59:12.007303 22305 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e6eedd88ce94e379a63aa9adb7c6e74 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:59:12.007501 22305 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e6eedd88ce94e379a63aa9adb7c6e74 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:59:12.009395 8450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 00:59:12.010350 22305 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e6eedd88ce94e379a63aa9adb7c6e74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e6eedd88ce94e379a63aa9adb7c6e74" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45719 } }
I20260924 00:59:12.010929 22305 leader_election.cc:302] T 00000000000000000000000000000000 P 5e6eedd88ce94e379a63aa9adb7c6e74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e6eedd88ce94e379a63aa9adb7c6e74; no voters:
I20260924 00:59:12.011571 22305 leader_election.cc:288] T 00000000000000000000000000000000 P 5e6eedd88ce94e379a63aa9adb7c6e74 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:59:12.011686 22310 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e6eedd88ce94e379a63aa9adb7c6e74 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:59:12.014072 22305 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e6eedd88ce94e379a63aa9adb7c6e74 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:59:12.014138 22310 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e6eedd88ce94e379a63aa9adb7c6e74 [term 1 LEADER]: Becoming Leader. State: Replica: 5e6eedd88ce94e379a63aa9adb7c6e74, State: Running, Role: LEADER
I20260924 00:59:12.014678 22310 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e6eedd88ce94e379a63aa9adb7c6e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e6eedd88ce94e379a63aa9adb7c6e74" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45719 } }
I20260924 00:59:12.017838 22312 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e6eedd88ce94e379a63aa9adb7c6e74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e6eedd88ce94e379a63aa9adb7c6e74. Latest consensus state: current_term: 1 leader_uuid: "5e6eedd88ce94e379a63aa9adb7c6e74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e6eedd88ce94e379a63aa9adb7c6e74" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45719 } } }
I20260924 00:59:12.017809 22311 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e6eedd88ce94e379a63aa9adb7c6e74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5e6eedd88ce94e379a63aa9adb7c6e74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e6eedd88ce94e379a63aa9adb7c6e74" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45719 } } }
I20260924 00:59:12.018278 22312 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e6eedd88ce94e379a63aa9adb7c6e74 [sys.catalog]: This master's current role is: LEADER
I20260924 00:59:12.018312 22311 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e6eedd88ce94e379a63aa9adb7c6e74 [sys.catalog]: This master's current role is: LEADER
I20260924 00:59:12.019804 22320 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:59:12.025077 22320 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:59:12.033629 22320 catalog_manager.cc:1347] Generated new cluster ID: 7b511e9abf1b489f9a03e5d880652a3b
I20260924 00:59:12.033779 22320 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:59:12.042970 22320 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:59:12.044824 22320 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:59:12.056030 22320 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e6eedd88ce94e379a63aa9adb7c6e74: Generated new TSK 0
I20260924 00:59:12.057070 22320 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 00:59:12.071267 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 22238
I20260924 00:59:12.072464 22300 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:59:12.163936 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 22238
2026-09-24T00:59:12Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1975 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:59:12 dist-test-slave-mmhs krb5kdc[22333](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:59:12 dist-test-slave-mmhs krb5kdc[22333](info): set up 2 sockets
Sep 24 00:59:12 dist-test-slave-mmhs krb5kdc[22333](info): commencing operation
krb5kdc: starting...
W20260924 00:59:13.292642 8450 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Sep 24 00:59:13 dist-test-slave-mmhs krb5kdc[22333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:59:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:59:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.64.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
I20260924 00:59:13.533109 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:45505
--webserver_interface=127.8.64.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:38521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:45505
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:59:13.860071 22346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:59:13.860409 22346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:59:13.860695 22346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:59:13.868655 22346 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:59:13.868814 22346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:59:13.868935 22346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:59:13.869053 22346 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:59:13.869133 22346 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:59:13.879508 22346 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:38521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:45505
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:45505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:59:13.880918 22346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:59:13.882346 22346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:59:13.894649 22351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:59:13.894773 22354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:59:13.894727 22352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:59:13.895496 22346 server_base.cc:1034] running on GCE node
I20260924 00:59:13.896313 22346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:59:13.897753 22346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:59:13.898957 22346 hybrid_clock.cc:648] HybridClock initialized: now 1790211553898909 us; error 32 us; skew 500 ppm
I20260924 00:59:13.899317 22346 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:59:13 dist-test-slave-mmhs krb5kdc[22333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211553, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:59:13.902379 22346 init.cc:377] Logged in from keytab as kudu/127.8.64.190@KRBTEST.COM (short username kudu)
I20260924 00:59:13.903807 22346 webserver.cc:460] Webserver started at http://127.8.64.190:33931/ using document root <none> and password file <none>
I20260924 00:59:13.904675 22346 fs_manager.cc:359] Metadata directory not provided
I20260924 00:59:13.904829 22346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:59:13.905133 22346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:59:13.908113 22346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "5f62630d137240a3b346349af92107ac"
format_stamp: "Formatted at 2026-09-24 00:59:13 on dist-test-slave-mmhs"
server_key: "b54bffb460091fa8d7c93f15610aa6fd"
server_key_iv: "5f9f7dd50abfd48e2f0afa2902d5362d"
server_key_version: "encryptionkey@0"
I20260924 00:59:13.908866 22346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "5f62630d137240a3b346349af92107ac"
format_stamp: "Formatted at 2026-09-24 00:59:13 on dist-test-slave-mmhs"
server_key: "b54bffb460091fa8d7c93f15610aa6fd"
server_key_iv: "5f9f7dd50abfd48e2f0afa2902d5362d"
server_key_version: "encryptionkey@0"
I20260924 00:59:13.913985 22346 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260924 00:59:13.917577 22361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:13.919037 22346 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 00:59:13.919245 22346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "5f62630d137240a3b346349af92107ac"
format_stamp: "Formatted at 2026-09-24 00:59:13 on dist-test-slave-mmhs"
server_key: "b54bffb460091fa8d7c93f15610aa6fd"
server_key_iv: "5f9f7dd50abfd48e2f0afa2902d5362d"
server_key_version: "encryptionkey@0"
I20260924 00:59:13.919450 22346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:59:13.945433 22346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:59:13.949412 22346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:59:13.949715 22346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:59:13.968107 22346 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.190:45505
I20260924 00:59:13.968156 22412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.190:45505 every 8 connection(s)
I20260924 00:59:13.969478 22346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 00:59:13.973069 22413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:59:13.976384 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 22346
I20260924 00:59:13.976704 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 00:59:13.977747 8450 external_mini_cluster.cc:1428] Setting key 9f61d59e4a233582fde3153f4b208cd7
I20260924 00:59:13.986868 22413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac: Bootstrap starting.
I20260924 00:59:13.990545 22413 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac: Neither blocks nor log segments found. Creating new log.
I20260924 00:59:13.991540 22413 log.cc:824] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac: Log is configured to *not* fsync() on all Append() calls
Sep 24 00:59:13 dist-test-slave-mmhs krb5kdc[22333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:59:13.994786 22413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac: No bootstrap required, opened a new log
I20260924 00:59:14.002028 22416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:13.979920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54248 (local address 127.8.64.190:45505)
0924 00:59:13.980741 (+ 821us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:13.980756 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:13.980805 (+ 49us) server_negotiation.cc:407] Connection header received
0924 00:59:13.982642 (+ 1837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:13.982670 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:13.983196 (+ 526us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:13.983625 (+ 429us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:13.984935 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:13.986902 (+ 1967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:13.988247 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:13.989322 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:13.989497 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:13.993473 (+ 3976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:13.993485 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:13.993504 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:13.993559 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:13.997567 (+ 4008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:13.998225 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:13.998237 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:13.998246 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:13.998459 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:13.998819 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:13.998824 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:13.998829 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:13.999573 (+ 744us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:13.999886 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:14.000201 (+ 315us) server_negotiation.cc:299] Negotiation successful
0924 00:59:14.000663 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":354,"threads_started":1}
I20260924 00:59:14.005326 22413 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f62630d137240a3b346349af92107ac" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45505 } }
I20260924 00:59:14.005635 22413 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:59:14.005712 22413 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f62630d137240a3b346349af92107ac, State: Initialized, Role: FOLLOWER
I20260924 00:59:14.006246 22413 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f62630d137240a3b346349af92107ac" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45505 } }
I20260924 00:59:14.006505 22413 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:59:14.006610 22413 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:59:14.006738 22413 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:59:14.009089 22413 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f62630d137240a3b346349af92107ac" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45505 } }
I20260924 00:59:14.009523 22413 leader_election.cc:302] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f62630d137240a3b346349af92107ac; no voters:
I20260924 00:59:14.010061 22413 leader_election.cc:288] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:59:14.010244 22418 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:59:14.011094 22418 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [term 1 LEADER]: Becoming Leader. State: Replica: 5f62630d137240a3b346349af92107ac, State: Running, Role: LEADER
I20260924 00:59:14.011651 22418 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f62630d137240a3b346349af92107ac" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45505 } }
I20260924 00:59:14.012192 22413 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:59:14.015125 22420 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f62630d137240a3b346349af92107ac. Latest consensus state: current_term: 1 leader_uuid: "5f62630d137240a3b346349af92107ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f62630d137240a3b346349af92107ac" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45505 } } }
I20260924 00:59:14.015687 22420 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [sys.catalog]: This master's current role is: LEADER
I20260924 00:59:14.015712 22419 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5f62630d137240a3b346349af92107ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f62630d137240a3b346349af92107ac" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45505 } } }
I20260924 00:59:14.016693 22419 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [sys.catalog]: This master's current role is: LEADER
I20260924 00:59:14.018060 22428 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:59:14.024016 22428 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:59:14.032914 22428 catalog_manager.cc:1347] Generated new cluster ID: d5f43b24f410479c892dd4e5c00bfd87
I20260924 00:59:14.033036 22428 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:59:14.040443 22428 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:59:14.041671 22428 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:59:14.058202 22428 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac: Generated new TSK 0
I20260924 00:59:14.059082 22428 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.64.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab.
Entry for principal kudu/127.8.64.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab.
I20260924 00:59:14.130400 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.129:0
--local_ip_for_outbound_sockets=127.8.64.129
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:45505
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab
--principal=kudu/127.8.64.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:38521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:59:14.456118 22439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:59:14.456398 22439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:59:14.456565 22439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:59:14.463753 22439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:59:14.463852 22439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:59:14.464062 22439 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.129
I20260924 00:59:14.473501 22439 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:38521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab
--principal=kudu/127.8.64.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:45505
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.129
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:59:14.474803 22439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:59:14.476097 22439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:59:14.488724 22445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:59:14.488961 22447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:59:14.488986 22444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:59:14.489476 22439 server_base.cc:1034] running on GCE node
I20260924 00:59:14.490078 22439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:59:14.491086 22439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:59:14.492327 22439 hybrid_clock.cc:648] HybridClock initialized: now 1790211554492262 us; error 59 us; skew 500 ppm
I20260924 00:59:14.492671 22439 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:59:14 dist-test-slave-mmhs krb5kdc[22333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211554, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:59:14.495815 22439 init.cc:377] Logged in from keytab as kudu/127.8.64.129@KRBTEST.COM (short username kudu)
I20260924 00:59:14.497304 22439 webserver.cc:460] Webserver started at http://127.8.64.129:33331/ using document root <none> and password file <none>
I20260924 00:59:14.497956 22439 fs_manager.cc:359] Metadata directory not provided
I20260924 00:59:14.498052 22439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:59:14.498286 22439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:59:14.501446 22439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-0/data/instance:
uuid: "471e118cd0fc492f936acf4f6258b095"
format_stamp: "Formatted at 2026-09-24 00:59:14 on dist-test-slave-mmhs"
server_key: "3eb2520249b6bc14e9ff1a51cd2887fc"
server_key_iv: "a7279037f157517a57403d81aca1653d"
server_key_version: "encryptionkey@0"
I20260924 00:59:14.502462 22439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance:
uuid: "471e118cd0fc492f936acf4f6258b095"
format_stamp: "Formatted at 2026-09-24 00:59:14 on dist-test-slave-mmhs"
server_key: "3eb2520249b6bc14e9ff1a51cd2887fc"
server_key_iv: "a7279037f157517a57403d81aca1653d"
server_key_version: "encryptionkey@0"
I20260924 00:59:14.508529 22439 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260924 00:59:14.511943 22454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:14.513490 22439 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260924 00:59:14.513684 22439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-0/wal
uuid: "471e118cd0fc492f936acf4f6258b095"
format_stamp: "Formatted at 2026-09-24 00:59:14 on dist-test-slave-mmhs"
server_key: "3eb2520249b6bc14e9ff1a51cd2887fc"
server_key_iv: "a7279037f157517a57403d81aca1653d"
server_key_version: "encryptionkey@0"
I20260924 00:59:14.513887 22439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:59:14.527413 22439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:59:14.531200 22439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:59:14.531483 22439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:59:14.532431 22439 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:59:14.533723 22439 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:59:14.533797 22439 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:14.533965 22439 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:59:14.534025 22439 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:14.568177 22439 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.129:40201
I20260924 00:59:14.568226 22567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.129:40201 every 8 connection(s)
I20260924 00:59:14.569515 22439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
I20260924 00:59:14.573154 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 22439
I20260924 00:59:14.573379 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance
I20260924 00:59:14.574132 8450 external_mini_cluster.cc:1428] Setting key 14987828639c963ec3d5307be702add6
Sep 24 00:59:14 dist-test-slave-mmhs krb5kdc[22333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211554, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:59:14.591148 22570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:14.572190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:40279 (local address 127.8.64.190:45505)
0924 00:59:14.572610 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:14.572614 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:14.573736 (+ 1122us) server_negotiation.cc:407] Connection header received
0924 00:59:14.576088 (+ 2352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:14.576094 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:14.576187 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:14.576286 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:14.577800 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:14.579025 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:14.580651 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:14.581397 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:14.581550 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:14.585338 (+ 3788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:14.585343 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:14.585346 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:14.585381 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:14.588066 (+ 2685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:14.588811 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:14.588817 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:14.588821 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:14.588943 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:14.589375 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:14.589381 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:14.589384 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:14.589646 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:14.589804 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:14.590541 (+ 737us) server_negotiation.cc:299] Negotiation successful
0924 00:59:14.590773 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":200,"threads_started":1}
I20260924 00:59:14.592087 22571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:14.572730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.129:40279)
0924 00:59:14.573560 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:14.573608 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0924 00:59:14.575757 (+ 2149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:59:14.576543 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:59:14.576551 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:59:14.577047 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:59:14.577612 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:14.577631 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:14.579181 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:14.579184 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:14.580505 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:14.580517 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:14.581715 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:14.581721 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:14.581933 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:14.582686 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:59:14.582711 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:59:14.585101 (+ 2390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:59:14.588237 (+ 3136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:59:14.588246 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:59:14.588264 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:59:14.588689 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:59:14.589058 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:59:14.589063 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:59:14.589067 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:59:14.589258 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:59:14.589773 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:59:14.589780 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:59:14.590121 (+ 341us) client_negotiation.cc:769] Sending connection context
0924 00:59:14.590434 (+ 313us) client_negotiation.cc:240] Negotiation successful
0924 00:59:14.590984 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":349,"threads_started":1}
I20260924 00:59:14.593644 22568 heartbeater.cc:347] Connected to a master server at 127.8.64.190:45505
I20260924 00:59:14.594024 22568 heartbeater.cc:464] Registering TS with master...
I20260924 00:59:14.595047 22568 heartbeater.cc:511] Master 127.8.64.190:45505 requested a full tablet report, sending...
I20260924 00:59:14.597339 22378 ts_manager.cc:194] Registered new tserver with Master: 471e118cd0fc492f936acf4f6258b095 (127.8.64.129:40201)
WARNING: no policy specified for kudu/127.8.64.130@KRBTEST.COM; defaulting to no policy
I20260924 00:59:14.598850 22378 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.129@KRBTEST.COM'} at 127.8.64.129:40279
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab.
Entry for principal kudu/127.8.64.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab.
I20260924 00:59:14.629918 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.130:0
--local_ip_for_outbound_sockets=127.8.64.130
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:45505
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab
--principal=kudu/127.8.64.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:38521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:59:14.951186 22574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:59:14.951450 22574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:59:14.951617 22574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:59:14.958830 22574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:59:14.958932 22574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:59:14.959118 22574 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.130
I20260924 00:59:14.969614 22574 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:38521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab
--principal=kudu/127.8.64.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:45505
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.130
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:59:14.971083 22574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:59:14.972693 22574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:59:14.987622 22583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:59:14.987604 22580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:59:14.988323 22574 server_base.cc:1034] running on GCE node
W20260924 00:59:14.988380 22579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:59:14.989002 22574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:59:14.989859 22574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:59:14.991082 22574 hybrid_clock.cc:648] HybridClock initialized: now 1790211554991039 us; error 27 us; skew 500 ppm
I20260924 00:59:14.991431 22574 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:59:14 dist-test-slave-mmhs krb5kdc[22333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211554, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:59:14.994590 22574 init.cc:377] Logged in from keytab as kudu/127.8.64.130@KRBTEST.COM (short username kudu)
I20260924 00:59:14.995952 22574 webserver.cc:460] Webserver started at http://127.8.64.130:41043/ using document root <none> and password file <none>
I20260924 00:59:14.996651 22574 fs_manager.cc:359] Metadata directory not provided
I20260924 00:59:14.996778 22574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:59:14.997077 22574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:59:14.999969 22574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-1/data/instance:
uuid: "a9b7028f039c46259584a9163095688a"
format_stamp: "Formatted at 2026-09-24 00:59:14 on dist-test-slave-mmhs"
server_key: "d243346bce3af47d0d00dd6e2ac8b335"
server_key_iv: "86bcea2c24db8983fa633cc9081cddc7"
server_key_version: "encryptionkey@0"
I20260924 00:59:15.000706 22574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance:
uuid: "a9b7028f039c46259584a9163095688a"
format_stamp: "Formatted at 2026-09-24 00:59:14 on dist-test-slave-mmhs"
server_key: "d243346bce3af47d0d00dd6e2ac8b335"
server_key_iv: "86bcea2c24db8983fa633cc9081cddc7"
server_key_version: "encryptionkey@0"
I20260924 00:59:15.005714 22574 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260924 00:59:15.009330 22590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:15.010690 22574 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 00:59:15.010860 22574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-1/wal
uuid: "a9b7028f039c46259584a9163095688a"
format_stamp: "Formatted at 2026-09-24 00:59:14 on dist-test-slave-mmhs"
server_key: "d243346bce3af47d0d00dd6e2ac8b335"
server_key_iv: "86bcea2c24db8983fa633cc9081cddc7"
server_key_version: "encryptionkey@0"
I20260924 00:59:15.011053 22574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:59:15.022859 22574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:59:15.027007 22574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:59:15.027302 22574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:59:15.028225 22574 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:59:15.029520 22574 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:59:15.029626 22574 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:15.029785 22574 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:59:15.029860 22574 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:15.063269 22574 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.130:40057
I20260924 00:59:15.063357 22702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.130:40057 every 8 connection(s)
I20260924 00:59:15.064558 22574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
I20260924 00:59:15.073330 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 22574
I20260924 00:59:15.073545 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance
I20260924 00:59:15.074018 8450 external_mini_cluster.cc:1428] Setting key f8691e41e410de57272af74400e2991f
Sep 24 00:59:15 dist-test-slave-mmhs krb5kdc[22333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211554, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.130@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:59:15.085223 22570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:15.067161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:53359 (local address 127.8.64.190:45505)
0924 00:59:15.067345 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:15.067350 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:15.068574 (+ 1224us) server_negotiation.cc:407] Connection header received
0924 00:59:15.070468 (+ 1894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:15.070474 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:15.070568 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:15.070687 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:15.072169 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:15.072868 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:15.074677 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:15.075183 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:15.075326 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:15.079306 (+ 3980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:15.079311 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:15.079314 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:15.079347 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:15.081962 (+ 2615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:15.082784 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:15.082793 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:15.082796 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:15.082917 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:15.083364 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:15.083369 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:15.083371 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:15.083640 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:15.083805 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:15.084759 (+ 954us) server_negotiation.cc:299] Negotiation successful
0924 00:59:15.084946 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260924 00:59:15.086683 22705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:15.067670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.130:53359)
0924 00:59:15.068412 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:15.068458 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0924 00:59:15.070141 (+ 1683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:59:15.070923 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:59:15.070931 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:59:15.071431 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:59:15.071993 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:15.072011 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:15.073072 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:15.073083 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:15.074525 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:15.074542 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:15.075404 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:15.075409 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:15.075617 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:15.076383 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:59:15.076413 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:59:15.079062 (+ 2649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:59:15.082109 (+ 3047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:59:15.082118 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:59:15.082135 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:59:15.082646 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:59:15.083029 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:59:15.083034 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:59:15.083038 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:59:15.083234 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:59:15.083792 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:59:15.083800 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:59:15.084289 (+ 489us) client_negotiation.cc:769] Sending connection context
0924 00:59:15.084612 (+ 323us) client_negotiation.cc:240] Negotiation successful
0924 00:59:15.085213 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":277,"threads_started":1}
I20260924 00:59:15.088204 22703 heartbeater.cc:347] Connected to a master server at 127.8.64.190:45505
I20260924 00:59:15.088537 22703 heartbeater.cc:464] Registering TS with master...
I20260924 00:59:15.089367 22703 heartbeater.cc:511] Master 127.8.64.190:45505 requested a full tablet report, sending...
I20260924 00:59:15.091074 22378 ts_manager.cc:194] Registered new tserver with Master: a9b7028f039c46259584a9163095688a (127.8.64.130:40057)
I20260924 00:59:15.091908 22378 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.130@KRBTEST.COM'} at 127.8.64.130:53359
WARNING: no policy specified for kudu/127.8.64.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab.
Entry for principal kudu/127.8.64.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab.
I20260924 00:59:15.131503 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.131:0
--local_ip_for_outbound_sockets=127.8.64.131
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:45505
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab
--principal=kudu/127.8.64.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:38521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:59:15.452301 22708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:59:15.452571 22708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:59:15.452665 22708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:59:15.459928 22708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:59:15.460047 22708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:59:15.460167 22708 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.131
I20260924 00:59:15.469732 22708 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:38521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab
--principal=kudu/127.8.64.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:45505
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.131
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:59:15.471029 22708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:59:15.472278 22708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:59:15.485055 22713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:59:15.485203 22714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:59:15.485553 22716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:59:15.486490 22708 server_base.cc:1034] running on GCE node
I20260924 00:59:15.487156 22708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:59:15.488140 22708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:59:15.489378 22708 hybrid_clock.cc:648] HybridClock initialized: now 1790211555489313 us; error 59 us; skew 500 ppm
I20260924 00:59:15.489722 22708 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:59:15 dist-test-slave-mmhs krb5kdc[22333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211555, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:59:15.492935 22708 init.cc:377] Logged in from keytab as kudu/127.8.64.131@KRBTEST.COM (short username kudu)
I20260924 00:59:15.494477 22708 webserver.cc:460] Webserver started at http://127.8.64.131:38823/ using document root <none> and password file <none>
I20260924 00:59:15.495158 22708 fs_manager.cc:359] Metadata directory not provided
I20260924 00:59:15.495298 22708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:59:15.495584 22708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:59:15.498458 22708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-2/data/instance:
uuid: "18e00ac71aed4617b1a6078ddc64d228"
format_stamp: "Formatted at 2026-09-24 00:59:15 on dist-test-slave-mmhs"
server_key: "c0a2219cbd355d4f8f49662f8f1e868c"
server_key_iv: "05b8aa2e023b621cf8fe3225a158b6f1"
server_key_version: "encryptionkey@0"
I20260924 00:59:15.499202 22708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance:
uuid: "18e00ac71aed4617b1a6078ddc64d228"
format_stamp: "Formatted at 2026-09-24 00:59:15 on dist-test-slave-mmhs"
server_key: "c0a2219cbd355d4f8f49662f8f1e868c"
server_key_iv: "05b8aa2e023b621cf8fe3225a158b6f1"
server_key_version: "encryptionkey@0"
I20260924 00:59:15.504135 22708 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260924 00:59:15.507547 22723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:15.508838 22708 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260924 00:59:15.509014 22708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-2/wal
uuid: "18e00ac71aed4617b1a6078ddc64d228"
format_stamp: "Formatted at 2026-09-24 00:59:15 on dist-test-slave-mmhs"
server_key: "c0a2219cbd355d4f8f49662f8f1e868c"
server_key_iv: "05b8aa2e023b621cf8fe3225a158b6f1"
server_key_version: "encryptionkey@0"
I20260924 00:59:15.509223 22708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:59:15.528764 22708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:59:15.533212 22708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:59:15.533499 22708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:59:15.534487 22708 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:59:15.535778 22708 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:59:15.535883 22708 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:15.536041 22708 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:59:15.536113 22708 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:15.569571 22708 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.131:35483
I20260924 00:59:15.569602 22836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.131:35483 every 8 connection(s)
I20260924 00:59:15.570848 22708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
I20260924 00:59:15.573678 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 22708
I20260924 00:59:15.573866 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance
I20260924 00:59:15.574365 8450 external_mini_cluster.cc:1428] Setting key ea880bb6971f7765a5634c05a534aca6
Sep 24 00:59:15 dist-test-slave-mmhs krb5kdc[22333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211555, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.131@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:59:15.590183 22570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:15.573459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:56493 (local address 127.8.64.190:45505)
0924 00:59:15.573703 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:15.573711 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:15.574968 (+ 1257us) server_negotiation.cc:407] Connection header received
0924 00:59:15.577086 (+ 2118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:15.577090 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:15.577152 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:15.577239 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:15.578761 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:15.579512 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:15.580986 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:15.581376 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:15.581464 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:15.584751 (+ 3287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:15.584756 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:15.584759 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:15.584789 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:15.587173 (+ 2384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:15.587928 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:15.587932 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:15.587934 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:15.588069 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:15.588507 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:15.588517 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:15.588519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:15.588784 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:15.588943 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:15.589611 (+ 668us) server_negotiation.cc:299] Negotiation successful
0924 00:59:15.589829 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260924 00:59:15.591161 22839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:15.573916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.131:56493)
0924 00:59:15.574793 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:15.574838 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0924 00:59:15.576817 (+ 1979us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:59:15.577481 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:59:15.577489 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:59:15.577995 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:59:15.578618 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:15.578635 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:15.579646 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:15.579649 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:15.580867 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:15.580878 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:15.581556 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:15.581560 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:15.581762 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:15.582458 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:59:15.582484 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:59:15.584575 (+ 2091us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:59:15.587310 (+ 2735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:59:15.587318 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:59:15.587334 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:59:15.587794 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:59:15.588192 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:59:15.588197 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:59:15.588201 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:59:15.588381 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:59:15.588915 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:59:15.588922 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:59:15.589256 (+ 334us) client_negotiation.cc:769] Sending connection context
0924 00:59:15.589510 (+ 254us) client_negotiation.cc:240] Negotiation successful
0924 00:59:15.590032 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":375,"threads_started":1}
I20260924 00:59:15.592456 22837 heartbeater.cc:347] Connected to a master server at 127.8.64.190:45505
I20260924 00:59:15.592758 22837 heartbeater.cc:464] Registering TS with master...
I20260924 00:59:15.593465 22837 heartbeater.cc:511] Master 127.8.64.190:45505 requested a full tablet report, sending...
I20260924 00:59:15.594863 22378 ts_manager.cc:194] Registered new tserver with Master: 18e00ac71aed4617b1a6078ddc64d228 (127.8.64.131:35483)
I20260924 00:59:15.595646 22378 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.131@KRBTEST.COM'} at 127.8.64.131:56493
I20260924 00:59:15.601884 8450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 00:59:15.601863 22568 heartbeater.cc:503] Master 127.8.64.190:45505 was elected leader, sending a full tablet report...
I20260924 00:59:15.619900 22570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:15.606671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45024 (local address 127.8.64.190:45505)
0924 00:59:15.606912 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:15.606922 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:15.607283 (+ 361us) server_negotiation.cc:407] Connection header received
0924 00:59:15.607402 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:15.607406 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:15.607468 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:15.607567 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:15.608848 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:15.609556 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:15.610946 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:15.611376 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:15.611480 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:15.613882 (+ 2402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:15.613887 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:15.613890 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:15.613920 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:15.616943 (+ 3023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:15.617706 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:15.617711 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:15.617713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:15.617858 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:15.618476 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:15.618480 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:15.618482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:15.618823 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:15.619034 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:15.619401 (+ 367us) server_negotiation.cc:299] Negotiation successful
0924 00:59:15.619560 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260924 00:59:15.623929 22378 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45024:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:59:15.630833 22378 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:59:15.663070 22849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:15.643075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.130:40057 (local address 127.0.0.1:56410)
0924 00:59:15.656396 (+ 13321us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:15.656447 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0924 00:59:15.656727 (+ 280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:59:15.657385 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:59:15.657393 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:59:15.657422 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:59:15.657799 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:15.657811 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:15.659193 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:15.659199 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:15.660711 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:15.660723 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:15.662034 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:15.662041 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:15.662271 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:15.662367 (+ 96us) client_negotiation.cc:769] Sending connection context
0924 00:59:15.662500 (+ 133us) client_negotiation.cc:240] Negotiation successful
0924 00:59:15.662692 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13068,"thread_start_us":12928,"threads_started":1}
I20260924 00:59:15.663064 22850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:15.643308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56410 (local address 127.8.64.130:40057)
0924 00:59:15.655579 (+ 12271us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:15.655593 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:15.656565 (+ 972us) server_negotiation.cc:407] Connection header received
0924 00:59:15.656961 (+ 396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:15.656969 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:15.657171 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:15.657395 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:59:15.657960 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:15.659058 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:15.660882 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:15.661826 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:15.661982 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:15.662194 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:15.662546 (+ 352us) server_negotiation.cc:299] Negotiation successful
0924 00:59:15.662709 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12038,"thread_start_us":12282,"threads_started":1}
I20260924 00:59:15.667157 22638 tablet_service.cc:1467] Processing CreateTablet for tablet dc62535aa82d404895ecdb2f54130886 (DEFAULT_TABLE table=test-table [id=5a119df84f1c4bd7bcb49d60f117944a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:59:15.669199 22638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc62535aa82d404895ecdb2f54130886. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:59:15.679684 22854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:15.644337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46954 (local address 127.8.64.131:35483)
0924 00:59:15.667189 (+ 22852us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:15.667202 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:15.668246 (+ 1044us) server_negotiation.cc:407] Connection header received
0924 00:59:15.668374 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:15.668385 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:15.668700 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:15.668919 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:59:15.670022 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:15.672236 (+ 2214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:15.676982 (+ 4746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:15.678492 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:15.678614 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:15.678848 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:15.679136 (+ 288us) server_negotiation.cc:299] Negotiation successful
0924 00:59:15.679308 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22616,"thread_start_us":22876,"threads_started":1}
I20260924 00:59:15.680030 22852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:15.643769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.131:35483 (local address 127.0.0.1:46954)
0924 00:59:15.667909 (+ 24140us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:15.667940 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0924 00:59:15.668093 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:59:15.669255 (+ 1162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:59:15.669263 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:59:15.669298 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:59:15.669756 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:15.669770 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:15.672443 (+ 2673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:15.672452 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:15.675362 (+ 2910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:15.675383 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:15.678712 (+ 3329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:15.678719 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:15.678913 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:15.678960 (+ 47us) client_negotiation.cc:769] Sending connection context
0924 00:59:15.679067 (+ 107us) client_negotiation.cc:240] Negotiation successful
0924 00:59:15.679651 (+ 584us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23981,"thread_start_us":31772,"threads_started":1}
I20260924 00:59:15.684414 22849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:15.643325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:40201 (local address 127.0.0.1:41866)
0924 00:59:15.663687 (+ 20362us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:15.663717 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0924 00:59:15.663815 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:59:15.677324 (+ 13509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:59:15.677331 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:59:15.677364 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:59:15.677572 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:15.677582 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:15.679440 (+ 1858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:15.679446 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:15.681404 (+ 1958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:15.681422 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:15.683608 (+ 2186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:15.683614 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:15.683803 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:15.683836 (+ 33us) client_negotiation.cc:769] Sending connection context
0924 00:59:15.683929 (+ 93us) client_negotiation.cc:240] Negotiation successful
0924 00:59:15.684100 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20263,"thread_start_us":32673,"threads_started":1}
I20260924 00:59:15.683835 22771 tablet_service.cc:1467] Processing CreateTablet for tablet dc62535aa82d404895ecdb2f54130886 (DEFAULT_TABLE table=test-table [id=5a119df84f1c4bd7bcb49d60f117944a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:59:15.684887 22853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:15.643975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41866 (local address 127.8.64.129:40201)
0924 00:59:15.676683 (+ 32708us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:15.676692 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:15.676721 (+ 29us) server_negotiation.cc:407] Connection header received
0924 00:59:15.676832 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:15.676842 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:15.677105 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:15.677313 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:59:15.677746 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:15.679269 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:15.681591 (+ 2322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:15.683115 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:15.683274 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:15.683506 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:15.684405 (+ 899us) server_negotiation.cc:299] Negotiation successful
0924 00:59:15.684579 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32526,"thread_start_us":32377,"threads_started":1}
I20260924 00:59:15.685408 22771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc62535aa82d404895ecdb2f54130886. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:59:15.687201 22855 tablet_bootstrap.cc:492] T dc62535aa82d404895ecdb2f54130886 P a9b7028f039c46259584a9163095688a: Bootstrap starting.
I20260924 00:59:15.688784 22503 tablet_service.cc:1467] Processing CreateTablet for tablet dc62535aa82d404895ecdb2f54130886 (DEFAULT_TABLE table=test-table [id=5a119df84f1c4bd7bcb49d60f117944a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:59:15.689898 22503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc62535aa82d404895ecdb2f54130886. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:59:15.692036 22855 tablet_bootstrap.cc:654] T dc62535aa82d404895ecdb2f54130886 P a9b7028f039c46259584a9163095688a: Neither blocks nor log segments found. Creating new log.
I20260924 00:59:15.693348 22855 log.cc:824] T dc62535aa82d404895ecdb2f54130886 P a9b7028f039c46259584a9163095688a: Log is configured to *not* fsync() on all Append() calls
I20260924 00:59:15.696606 22855 tablet_bootstrap.cc:492] T dc62535aa82d404895ecdb2f54130886 P a9b7028f039c46259584a9163095688a: No bootstrap required, opened a new log
I20260924 00:59:15.696880 22855 ts_tablet_manager.cc:1397] T dc62535aa82d404895ecdb2f54130886 P a9b7028f039c46259584a9163095688a: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260924 00:59:15.700567 22857 tablet_bootstrap.cc:492] T dc62535aa82d404895ecdb2f54130886 P 18e00ac71aed4617b1a6078ddc64d228: Bootstrap starting.
I20260924 00:59:15.702099 22858 tablet_bootstrap.cc:492] T dc62535aa82d404895ecdb2f54130886 P 471e118cd0fc492f936acf4f6258b095: Bootstrap starting.
I20260924 00:59:15.704123 22857 tablet_bootstrap.cc:654] T dc62535aa82d404895ecdb2f54130886 P 18e00ac71aed4617b1a6078ddc64d228: Neither blocks nor log segments found. Creating new log.
I20260924 00:59:15.705368 22857 log.cc:824] T dc62535aa82d404895ecdb2f54130886 P 18e00ac71aed4617b1a6078ddc64d228: Log is configured to *not* fsync() on all Append() calls
I20260924 00:59:15.705580 22858 tablet_bootstrap.cc:654] T dc62535aa82d404895ecdb2f54130886 P 471e118cd0fc492f936acf4f6258b095: Neither blocks nor log segments found. Creating new log.
I20260924 00:59:15.706795 22858 log.cc:824] T dc62535aa82d404895ecdb2f54130886 P 471e118cd0fc492f936acf4f6258b095: Log is configured to *not* fsync() on all Append() calls
I20260924 00:59:15.708418 22855 raft_consensus.cc:348] T dc62535aa82d404895ecdb2f54130886 P a9b7028f039c46259584a9163095688a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9b7028f039c46259584a9163095688a" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 40057 } } peers { permanent_uuid: "471e118cd0fc492f936acf4f6258b095" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 40201 } } peers { permanent_uuid: "18e00ac71aed4617b1a6078ddc64d228" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35483 } }
I20260924 00:59:15.709009 22855 raft_consensus.cc:374] T dc62535aa82d404895ecdb2f54130886 P a9b7028f039c46259584a9163095688a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:59:15.709127 22855 raft_consensus.cc:729] T dc62535aa82d404895ecdb2f54130886 P a9b7028f039c46259584a9163095688a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9b7028f039c46259584a9163095688a, State: Initialized, Role: FOLLOWER
I20260924 00:59:15.709909 22855 consensus_queue.cc:260] T dc62535aa82d404895ecdb2f54130886 P a9b7028f039c46259584a9163095688a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9b7028f039c46259584a9163095688a" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 40057 } } peers { permanent_uuid: "471e118cd0fc492f936acf4f6258b095" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 40201 } } peers { permanent_uuid: "18e00ac71aed4617b1a6078ddc64d228" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35483 } }
I20260924 00:59:15.711220 22703 heartbeater.cc:503] Master 127.8.64.190:45505 was elected leader, sending a full tablet report...
I20260924 00:59:15.712677 22855 ts_tablet_manager.cc:1428] T dc62535aa82d404895ecdb2f54130886 P a9b7028f039c46259584a9163095688a: Time spent starting tablet: real 0.016s user 0.012s sys 0.005s
I20260924 00:59:15.713294 22858 tablet_bootstrap.cc:492] T dc62535aa82d404895ecdb2f54130886 P 471e118cd0fc492f936acf4f6258b095: No bootstrap required, opened a new log
I20260924 00:59:15.713554 22858 ts_tablet_manager.cc:1397] T dc62535aa82d404895ecdb2f54130886 P 471e118cd0fc492f936acf4f6258b095: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.003s
I20260924 00:59:15.714527 22857 tablet_bootstrap.cc:492] T dc62535aa82d404895ecdb2f54130886 P 18e00ac71aed4617b1a6078ddc64d228: No bootstrap required, opened a new log
I20260924 00:59:15.714848 22857 ts_tablet_manager.cc:1397] T dc62535aa82d404895ecdb2f54130886 P 18e00ac71aed4617b1a6078ddc64d228: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.007s
I20260924 00:59:15.721454 22858 raft_consensus.cc:348] T dc62535aa82d404895ecdb2f54130886 P 471e118cd0fc492f936acf4f6258b095 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9b7028f039c46259584a9163095688a" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 40057 } } peers { permanent_uuid: "471e118cd0fc492f936acf4f6258b095" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 40201 } } peers { permanent_uuid: "18e00ac71aed4617b1a6078ddc64d228" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35483 } }
I20260924 00:59:15.721820 22858 raft_consensus.cc:374] T dc62535aa82d404895ecdb2f54130886 P 471e118cd0fc492f936acf4f6258b095 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:59:15.721925 22858 raft_consensus.cc:729] T dc62535aa82d404895ecdb2f54130886 P 471e118cd0fc492f936acf4f6258b095 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 471e118cd0fc492f936acf4f6258b095, State: Initialized, Role: FOLLOWER
I20260924 00:59:15.722568 22858 consensus_queue.cc:260] T dc62535aa82d404895ecdb2f54130886 P 471e118cd0fc492f936acf4f6258b095 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9b7028f039c46259584a9163095688a" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 40057 } } peers { permanent_uuid: "471e118cd0fc492f936acf4f6258b095" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 40201 } } peers { permanent_uuid: "18e00ac71aed4617b1a6078ddc64d228" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35483 } }
I20260924 00:59:15.723297 22857 raft_consensus.cc:348] T dc62535aa82d404895ecdb2f54130886 P 18e00ac71aed4617b1a6078ddc64d228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9b7028f039c46259584a9163095688a" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 40057 } } peers { permanent_uuid: "471e118cd0fc492f936acf4f6258b095" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 40201 } } peers { permanent_uuid: "18e00ac71aed4617b1a6078ddc64d228" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35483 } }
I20260924 00:59:15.723930 22857 raft_consensus.cc:374] T dc62535aa82d404895ecdb2f54130886 P 18e00ac71aed4617b1a6078ddc64d228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:59:15.724063 22857 raft_consensus.cc:729] T dc62535aa82d404895ecdb2f54130886 P 18e00ac71aed4617b1a6078ddc64d228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18e00ac71aed4617b1a6078ddc64d228, State: Initialized, Role: FOLLOWER
I20260924 00:59:15.724251 22858 ts_tablet_manager.cc:1428] T dc62535aa82d404895ecdb2f54130886 P 471e118cd0fc492f936acf4f6258b095: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260924 00:59:15.724637 22857 consensus_queue.cc:260] T dc62535aa82d404895ecdb2f54130886 P 18e00ac71aed4617b1a6078ddc64d228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9b7028f039c46259584a9163095688a" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 40057 } } peers { permanent_uuid: "471e118cd0fc492f936acf4f6258b095" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 40201 } } peers { permanent_uuid: "18e00ac71aed4617b1a6078ddc64d228" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35483 } }
I20260924 00:59:15.725618 22837 heartbeater.cc:503] Master 127.8.64.190:45505 was elected leader, sending a full tablet report...
I20260924 00:59:15.726377 22857 ts_tablet_manager.cc:1428] T dc62535aa82d404895ecdb2f54130886 P 18e00ac71aed4617b1a6078ddc64d228: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
W20260924 00:59:15.816572 22704 tablet.cc:2369] T dc62535aa82d404895ecdb2f54130886 P a9b7028f039c46259584a9163095688a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 00:59:15.821888 22569 tablet.cc:2369] T dc62535aa82d404895ecdb2f54130886 P 471e118cd0fc492f936acf4f6258b095: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 00:59:15.822515 22838 tablet.cc:2369] T dc62535aa82d404895ecdb2f54130886 P 18e00ac71aed4617b1a6078ddc64d228: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:59:15.904184 22861 raft_consensus.cc:482] T dc62535aa82d404895ecdb2f54130886 P a9b7028f039c46259584a9163095688a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:59:15.904528 22861 raft_consensus.cc:504] T dc62535aa82d404895ecdb2f54130886 P a9b7028f039c46259584a9163095688a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9b7028f039c46259584a9163095688a" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 40057 } } peers { permanent_uuid: "471e118cd0fc492f936acf4f6258b095" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 40201 } } peers { permanent_uuid: "18e00ac71aed4617b1a6078ddc64d228" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35483 } }
I20260924 00:59:15.906373 22861 leader_election.cc:288] T dc62535aa82d404895ecdb2f54130886 P a9b7028f039c46259584a9163095688a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 471e118cd0fc492f936acf4f6258b095 (127.8.64.129:40201), 18e00ac71aed4617b1a6078ddc64d228 (127.8.64.131:35483)
I20260924 00:59:15.913869 22854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:15.906938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:60353 (local address 127.8.64.131:35483)
0924 00:59:15.907120 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:15.907128 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:15.907813 (+ 685us) server_negotiation.cc:407] Connection header received
0924 00:59:15.908092 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:15.908097 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:15.908202 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:15.908291 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:59:15.909021 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:15.909765 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:15.911785 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:15.912946 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:15.913107 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:15.913177 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:15.913483 (+ 306us) server_negotiation.cc:299] Negotiation successful
0924 00:59:15.913609 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260924 00:59:15.914044 22853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:15.906895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:57753 (local address 127.8.64.129:40201)
0924 00:59:15.907108 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:15.907115 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:15.907726 (+ 611us) server_negotiation.cc:407] Connection header received
0924 00:59:15.907988 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:15.907995 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:15.908080 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:15.908144 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:59:15.909085 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:15.909777 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:15.911720 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:15.912887 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:15.913041 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:15.913113 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:15.913658 (+ 545us) server_negotiation.cc:299] Negotiation successful
0924 00:59:15.913780 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260924 00:59:15.914178 22865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:15.906656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.131:35483 (local address 127.8.64.130:60353)
0924 00:59:15.907702 (+ 1046us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:15.907723 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0924 00:59:15.907932 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:59:15.908327 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:59:15.908359 (+ 32us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:59:15.908389 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:59:15.908842 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:15.908852 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:15.909897 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:15.909904 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:15.911606 (+ 1702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:15.911618 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:15.913136 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:15.913142 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:15.913317 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:15.913347 (+ 30us) client_negotiation.cc:769] Sending connection context
0924 00:59:15.913462 (+ 115us) client_negotiation.cc:240] Negotiation successful
0924 00:59:15.913589 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":872,"mutex_wait_us":99,"thread_start_us":555,"threads_started":1}
I20260924 00:59:15.914315 22864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:15.906639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:40201 (local address 127.8.64.130:57753)
0924 00:59:15.907576 (+ 937us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:15.907598 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0924 00:59:15.907797 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:59:15.908359 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:59:15.908364 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:59:15.908487 (+ 123us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:59:15.908922 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:15.908931 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:15.909914 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:15.909920 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:15.911560 (+ 1640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:15.911572 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:15.913318 (+ 1746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:15.913324 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:15.913502 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:15.913526 (+ 24us) client_negotiation.cc:769] Sending connection context
0924 00:59:15.913631 (+ 105us) client_negotiation.cc:240] Negotiation successful
0924 00:59:15.913765 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":785,"mutex_wait_us":6,"spinlock_wait_cycles":24448,"thread_start_us":1139,"threads_started":1}
I20260924 00:59:15.915323 22791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc62535aa82d404895ecdb2f54130886" candidate_uuid: "a9b7028f039c46259584a9163095688a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18e00ac71aed4617b1a6078ddc64d228" is_pre_election: true
I20260924 00:59:15.915323 22523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc62535aa82d404895ecdb2f54130886" candidate_uuid: "a9b7028f039c46259584a9163095688a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "471e118cd0fc492f936acf4f6258b095" is_pre_election: true
I20260924 00:59:15.915922 22791 raft_consensus.cc:2454] T dc62535aa82d404895ecdb2f54130886 P 18e00ac71aed4617b1a6078ddc64d228 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9b7028f039c46259584a9163095688a in term 0.
I20260924 00:59:15.915922 22523 raft_consensus.cc:2454] T dc62535aa82d404895ecdb2f54130886 P 471e118cd0fc492f936acf4f6258b095 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9b7028f039c46259584a9163095688a in term 0.
I20260924 00:59:15.916658 22591 leader_election.cc:302] T dc62535aa82d404895ecdb2f54130886 P a9b7028f039c46259584a9163095688a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 18e00ac71aed4617b1a6078ddc64d228, 471e118cd0fc492f936acf4f6258b095, a9b7028f039c46259584a9163095688a; no voters:
I20260924 00:59:15.917222 22861 raft_consensus.cc:2781] T dc62535aa82d404895ecdb2f54130886 P a9b7028f039c46259584a9163095688a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:59:15.917418 22861 raft_consensus.cc:482] T dc62535aa82d404895ecdb2f54130886 P a9b7028f039c46259584a9163095688a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:59:15.917523 22861 raft_consensus.cc:3037] T dc62535aa82d404895ecdb2f54130886 P a9b7028f039c46259584a9163095688a [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:59:15.919847 22861 raft_consensus.cc:504] T dc62535aa82d404895ecdb2f54130886 P a9b7028f039c46259584a9163095688a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9b7028f039c46259584a9163095688a" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 40057 } } peers { permanent_uuid: "471e118cd0fc492f936acf4f6258b095" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 40201 } } peers { permanent_uuid: "18e00ac71aed4617b1a6078ddc64d228" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35483 } }
I20260924 00:59:15.920372 22861 leader_election.cc:288] T dc62535aa82d404895ecdb2f54130886 P a9b7028f039c46259584a9163095688a [CANDIDATE]: Term 1 election: Requested vote from peers 471e118cd0fc492f936acf4f6258b095 (127.8.64.129:40201), 18e00ac71aed4617b1a6078ddc64d228 (127.8.64.131:35483)
I20260924 00:59:15.920864 22523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc62535aa82d404895ecdb2f54130886" candidate_uuid: "a9b7028f039c46259584a9163095688a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "471e118cd0fc492f936acf4f6258b095"
I20260924 00:59:15.920866 22791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc62535aa82d404895ecdb2f54130886" candidate_uuid: "a9b7028f039c46259584a9163095688a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18e00ac71aed4617b1a6078ddc64d228"
I20260924 00:59:15.921079 22791 raft_consensus.cc:3037] T dc62535aa82d404895ecdb2f54130886 P 18e00ac71aed4617b1a6078ddc64d228 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:59:15.921094 22523 raft_consensus.cc:3037] T dc62535aa82d404895ecdb2f54130886 P 471e118cd0fc492f936acf4f6258b095 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:59:15.924463 22523 raft_consensus.cc:2454] T dc62535aa82d404895ecdb2f54130886 P 471e118cd0fc492f936acf4f6258b095 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9b7028f039c46259584a9163095688a in term 1.
I20260924 00:59:15.924463 22791 raft_consensus.cc:2454] T dc62535aa82d404895ecdb2f54130886 P 18e00ac71aed4617b1a6078ddc64d228 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9b7028f039c46259584a9163095688a in term 1.
I20260924 00:59:15.925222 22593 leader_election.cc:302] T dc62535aa82d404895ecdb2f54130886 P a9b7028f039c46259584a9163095688a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18e00ac71aed4617b1a6078ddc64d228, a9b7028f039c46259584a9163095688a; no voters:
I20260924 00:59:15.925630 22861 raft_consensus.cc:2781] T dc62535aa82d404895ecdb2f54130886 P a9b7028f039c46259584a9163095688a [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:59:15.926460 22861 raft_consensus.cc:686] T dc62535aa82d404895ecdb2f54130886 P a9b7028f039c46259584a9163095688a [term 1 LEADER]: Becoming Leader. State: Replica: a9b7028f039c46259584a9163095688a, State: Running, Role: LEADER
I20260924 00:59:15.927052 22861 consensus_queue.cc:237] T dc62535aa82d404895ecdb2f54130886 P a9b7028f039c46259584a9163095688a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9b7028f039c46259584a9163095688a" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 40057 } } peers { permanent_uuid: "471e118cd0fc492f936acf4f6258b095" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 40201 } } peers { permanent_uuid: "18e00ac71aed4617b1a6078ddc64d228" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35483 } }
I20260924 00:59:15.935292 22377 catalog_manager.cc:5495] T dc62535aa82d404895ecdb2f54130886 P a9b7028f039c46259584a9163095688a reported cstate change: term changed from 0 to 1, leader changed from <none> to a9b7028f039c46259584a9163095688a (127.8.64.130). New cstate: current_term: 1 leader_uuid: "a9b7028f039c46259584a9163095688a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9b7028f039c46259584a9163095688a" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 40057 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "471e118cd0fc492f936acf4f6258b095" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 40201 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18e00ac71aed4617b1a6078ddc64d228" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35483 } health_report { overall_health: UNKNOWN } } }
I20260924 00:59:15.984193 22850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:15.977853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56424 (local address 127.8.64.130:40057)
0924 00:59:15.978089 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:15.978096 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:15.978118 (+ 22us) server_negotiation.cc:407] Connection header received
0924 00:59:15.978225 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:15.978231 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:15.978376 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:15.978541 (+ 165us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:59:15.978880 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:15.980076 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:15.981526 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:15.982221 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:15.982382 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:15.982848 (+ 466us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:59:15.983552 (+ 704us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:59:15.983706 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:15.983786 (+ 80us) server_negotiation.cc:299] Negotiation successful
0924 00:59:15.983873 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260924 00:59:15.993535 22791 raft_consensus.cc:1261] T dc62535aa82d404895ecdb2f54130886 P 18e00ac71aed4617b1a6078ddc64d228 [term 1 FOLLOWER]: Refusing update from remote peer a9b7028f039c46259584a9163095688a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:59:15.993535 22523 raft_consensus.cc:1261] T dc62535aa82d404895ecdb2f54130886 P 471e118cd0fc492f936acf4f6258b095 [term 1 FOLLOWER]: Refusing update from remote peer a9b7028f039c46259584a9163095688a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:59:15.994628 22866 consensus_queue.cc:1035] T dc62535aa82d404895ecdb2f54130886 P a9b7028f039c46259584a9163095688a [LEADER]: Connected to new peer: Peer: permanent_uuid: "471e118cd0fc492f936acf4f6258b095" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 40201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:59:15.994978 22861 consensus_queue.cc:1035] T dc62535aa82d404895ecdb2f54130886 P a9b7028f039c46259584a9163095688a [LEADER]: Connected to new peer: Peer: permanent_uuid: "18e00ac71aed4617b1a6078ddc64d228" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:59:16.006532 22869 mvcc.cc:205] Tried to move back new op lower bound from 7332706533338890240 to 7332706533089112064. Current Snapshot: MvccSnapshot[applied={T|T < 7332706533338890240}]
I20260924 00:59:16.007290 22870 mvcc.cc:205] Tried to move back new op lower bound from 7332706533338890240 to 7332706533089112064. Current Snapshot: MvccSnapshot[applied={T|T < 7332706533338890240}]
I20260924 00:59:16.021808 22378 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:59:16.022163 22378 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:59:16.026163 22378 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac: Sending DeleteTablet for 3 replicas of tablet dc62535aa82d404895ecdb2f54130886
I20260924 00:59:16.027431 22503 tablet_service.cc:1514] Processing DeleteTablet for tablet dc62535aa82d404895ecdb2f54130886 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:59:16 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:41866
I20260924 00:59:16.027468 22638 tablet_service.cc:1514] Processing DeleteTablet for tablet dc62535aa82d404895ecdb2f54130886 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:59:16 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:56410
I20260924 00:59:16.028384 22879 tablet_replica.cc:330] T dc62535aa82d404895ecdb2f54130886 P 471e118cd0fc492f936acf4f6258b095: stopping tablet replica
I20260924 00:59:16.028496 22771 tablet_service.cc:1514] Processing DeleteTablet for tablet dc62535aa82d404895ecdb2f54130886 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:59:16 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:46954
I20260924 00:59:16.028623 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 22439
I20260924 00:59:16.028857 22879 raft_consensus.cc:2229] T dc62535aa82d404895ecdb2f54130886 P 471e118cd0fc492f936acf4f6258b095 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:59:16.029413 22879 raft_consensus.cc:2258] T dc62535aa82d404895ecdb2f54130886 P 471e118cd0fc492f936acf4f6258b095 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:59:16.032006 22879 ts_tablet_manager.cc:1905] T dc62535aa82d404895ecdb2f54130886 P 471e118cd0fc492f936acf4f6258b095: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:59:16.040767 22879 ts_tablet_manager.cc:1918] T dc62535aa82d404895ecdb2f54130886 P 471e118cd0fc492f936acf4f6258b095: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:59:16.041054 22879 log.cc:1196] T dc62535aa82d404895ecdb2f54130886 P 471e118cd0fc492f936acf4f6258b095: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-0/wal/wals/dc62535aa82d404895ecdb2f54130886
I20260924 00:59:16.041294 22881 tablet_replica.cc:330] T dc62535aa82d404895ecdb2f54130886 P 18e00ac71aed4617b1a6078ddc64d228: stopping tablet replica
I20260924 00:59:16.041723 22879 ts_tablet_manager.cc:1939] T dc62535aa82d404895ecdb2f54130886 P 471e118cd0fc492f936acf4f6258b095: Deleting consensus metadata
I20260924 00:59:16.041927 22881 raft_consensus.cc:2229] T dc62535aa82d404895ecdb2f54130886 P 18e00ac71aed4617b1a6078ddc64d228 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:59:16.042526 22881 raft_consensus.cc:2258] T dc62535aa82d404895ecdb2f54130886 P 18e00ac71aed4617b1a6078ddc64d228 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:59:16.044724 22881 ts_tablet_manager.cc:1905] T dc62535aa82d404895ecdb2f54130886 P 18e00ac71aed4617b1a6078ddc64d228: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:59:16.044654 22362 catalog_manager.cc:4841] TS 471e118cd0fc492f936acf4f6258b095 (127.8.64.129:40201): tablet dc62535aa82d404895ecdb2f54130886 (table test-table [id=5a119df84f1c4bd7bcb49d60f117944a]) successfully deleted
I20260924 00:59:16.050546 22880 tablet_replica.cc:330] T dc62535aa82d404895ecdb2f54130886 P a9b7028f039c46259584a9163095688a: stopping tablet replica
I20260924 00:59:16.050958 22880 raft_consensus.cc:2229] T dc62535aa82d404895ecdb2f54130886 P a9b7028f039c46259584a9163095688a [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:59:16.051520 22880 raft_consensus.cc:2258] T dc62535aa82d404895ecdb2f54130886 P a9b7028f039c46259584a9163095688a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:59:16.051718 22881 ts_tablet_manager.cc:1918] T dc62535aa82d404895ecdb2f54130886 P 18e00ac71aed4617b1a6078ddc64d228: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:59:16.051882 22881 log.cc:1196] T dc62535aa82d404895ecdb2f54130886 P 18e00ac71aed4617b1a6078ddc64d228: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-2/wal/wals/dc62535aa82d404895ecdb2f54130886
I20260924 00:59:16.052373 22881 ts_tablet_manager.cc:1939] T dc62535aa82d404895ecdb2f54130886 P 18e00ac71aed4617b1a6078ddc64d228: Deleting consensus metadata
I20260924 00:59:16.053900 22880 ts_tablet_manager.cc:1905] T dc62535aa82d404895ecdb2f54130886 P a9b7028f039c46259584a9163095688a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:59:16.054330 22364 catalog_manager.cc:4841] TS 18e00ac71aed4617b1a6078ddc64d228 (127.8.64.131:35483): tablet dc62535aa82d404895ecdb2f54130886 (table test-table [id=5a119df84f1c4bd7bcb49d60f117944a]) successfully deleted
Sep 24 00:59:16 dist-test-slave-mmhs krb5kdc[22333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.129@KRBTEST.COM
I20260924 00:59:16.061072 22880 ts_tablet_manager.cc:1918] T dc62535aa82d404895ecdb2f54130886 P a9b7028f039c46259584a9163095688a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:59:16.061255 22880 log.cc:1196] T dc62535aa82d404895ecdb2f54130886 P a9b7028f039c46259584a9163095688a: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-1/wal/wals/dc62535aa82d404895ecdb2f54130886
I20260924 00:59:16.061906 22880 ts_tablet_manager.cc:1939] T dc62535aa82d404895ecdb2f54130886 P a9b7028f039c46259584a9163095688a: Deleting consensus metadata
I20260924 00:59:16.063920 22365 catalog_manager.cc:4841] TS a9b7028f039c46259584a9163095688a (127.8.64.130:40057): tablet dc62535aa82d404895ecdb2f54130886 (table test-table [id=5a119df84f1c4bd7bcb49d60f117944a]) successfully deleted
I20260924 00:59:16.067981 22853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:16.029741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41870 (local address 127.8.64.129:40201)
0924 00:59:16.029920 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:16.029926 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:16.046944 (+ 17018us) server_negotiation.cc:407] Connection header received
0924 00:59:16.047293 (+ 349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:16.047302 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:16.047412 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:16.047497 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:16.051949 (+ 4452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:16.053026 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:16.055134 (+ 2108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:16.055779 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:16.055928 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:16.061559 (+ 5631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:16.061579 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:16.061600 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:16.061658 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:16.065027 (+ 3369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:16.065767 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:16.065774 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:16.065776 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:16.065905 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:16.066330 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:16.066338 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:16.066344 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:16.066705 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:16.067015 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:16.067378 (+ 363us) server_negotiation.cc:299] Negotiation successful
0924 00:59:16.067587 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260924 00:59:16.068639 22563 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:59:16.159909 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 22439
I20260924 00:59:16.184718 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 22574
Sep 24 00:59:16 dist-test-slave-mmhs krb5kdc[22333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.130@KRBTEST.COM
I20260924 00:59:16.202512 22850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:16.185527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56428 (local address 127.8.64.130:40057)
0924 00:59:16.185707 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:16.185711 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:16.185765 (+ 54us) server_negotiation.cc:407] Connection header received
0924 00:59:16.185958 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:16.185962 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:16.186033 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:16.186123 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:16.187835 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:16.188734 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:16.190701 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:16.191172 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:16.191274 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:16.196515 (+ 5241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:16.196529 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:16.196544 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:16.196584 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:16.199626 (+ 3042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:16.200485 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:16.200491 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:16.200493 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:16.200656 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:16.201047 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:16.201050 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:16.201052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:16.201402 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:16.201657 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:16.201978 (+ 321us) server_negotiation.cc:299] Negotiation successful
0924 00:59:16.202131 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260924 00:59:16.203290 22698 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:59:16.313730 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 22574
I20260924 00:59:16.332664 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 22708
Sep 24 00:59:16 dist-test-slave-mmhs krb5kdc[22333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.131@KRBTEST.COM
I20260924 00:59:16.347720 22854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:16.333421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46962 (local address 127.8.64.131:35483)
0924 00:59:16.333598 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:16.333603 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:16.333621 (+ 18us) server_negotiation.cc:407] Connection header received
0924 00:59:16.333756 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:16.333760 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:16.333826 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:16.333906 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:16.335268 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:16.335978 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:16.337220 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:16.337583 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:16.337680 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:16.342161 (+ 4481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:16.342175 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:16.342191 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:16.342225 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:16.344806 (+ 2581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:16.345505 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:16.345512 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:16.345514 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:16.345652 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:16.346055 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:16.346059 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:16.346062 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:16.346517 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:16.346813 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:16.347143 (+ 330us) server_negotiation.cc:299] Negotiation successful
0924 00:59:16.347362 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260924 00:59:16.348379 22832 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:59:16.436136 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 22708
I20260924 00:59:16.453006 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 22346
I20260924 00:59:16.454111 22408 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:59:16.563323 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 22346
I20260924 00:59:16.585036 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:45505
--webserver_interface=127.8.64.190
--webserver_port=33931
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:38521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:45505
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:59:16.911942 22887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:59:16.912245 22887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:59:16.912408 22887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:59:16.919936 22887 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:59:16.920063 22887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:59:16.920159 22887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:59:16.920248 22887 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:59:16.920337 22887 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:59:16.930601 22887 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:38521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:45505
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:45505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=33931
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:59:16.931957 22887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:59:16.933290 22887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:59:16.955741 22893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:59:16.955807 22896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:59:16.955804 22894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:59:16.957060 22887 server_base.cc:1034] running on GCE node
I20260924 00:59:16.957927 22887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:59:16.959518 22887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:59:16.960767 22887 hybrid_clock.cc:648] HybridClock initialized: now 1790211556960709 us; error 68 us; skew 500 ppm
I20260924 00:59:16.961179 22887 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:59:16 dist-test-slave-mmhs krb5kdc[22333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211556, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:59:16.964754 22887 init.cc:377] Logged in from keytab as kudu/127.8.64.190@KRBTEST.COM (short username kudu)
I20260924 00:59:16.966497 22887 webserver.cc:460] Webserver started at http://127.8.64.190:33931/ using document root <none> and password file <none>
I20260924 00:59:16.967236 22887 fs_manager.cc:359] Metadata directory not provided
I20260924 00:59:16.967401 22887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:59:16.973197 22887 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260924 00:59:16.976142 22903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:16.977483 22887 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260924 00:59:16.977687 22887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "5f62630d137240a3b346349af92107ac"
format_stamp: "Formatted at 2026-09-24 00:59:13 on dist-test-slave-mmhs"
server_key: "b54bffb460091fa8d7c93f15610aa6fd"
server_key_iv: "5f9f7dd50abfd48e2f0afa2902d5362d"
server_key_version: "encryptionkey@0"
I20260924 00:59:16.978807 22887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:59:17.001024 22887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:59:17.005788 22887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:59:17.006076 22887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:59:17.025188 22887 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.190:45505
I20260924 00:59:17.025195 22955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.190:45505 every 8 connection(s)
I20260924 00:59:17.026804 22887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 00:59:17.028285 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 22887
I20260924 00:59:17.029346 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.129:40201
--local_ip_for_outbound_sockets=127.8.64.129
--tserver_master_addrs=127.8.64.190:45505
--webserver_port=33331
--webserver_interface=127.8.64.129
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab
--principal=kudu/127.8.64.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:38521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 00:59:17.037294 22956 sys_catalog.cc:263] Verifying existing consensus state
I20260924 00:59:17.041106 22956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac: Bootstrap starting.
I20260924 00:59:17.061256 22956 log.cc:824] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac: Log is configured to *not* fsync() on all Append() calls
I20260924 00:59:17.073423 22956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 00:59:17.073987 22956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac: Bootstrap complete.
I20260924 00:59:17.083698 22956 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f62630d137240a3b346349af92107ac" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45505 } }
I20260924 00:59:17.084539 22956 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f62630d137240a3b346349af92107ac, State: Initialized, Role: FOLLOWER
I20260924 00:59:17.085156 22956 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f62630d137240a3b346349af92107ac" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45505 } }
I20260924 00:59:17.085397 22956 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:59:17.085518 22956 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:59:17.085721 22956 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [term 1 FOLLOWER]: Advancing to term 2
I20260924 00:59:17.089623 22956 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f62630d137240a3b346349af92107ac" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45505 } }
I20260924 00:59:17.090155 22956 leader_election.cc:302] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f62630d137240a3b346349af92107ac; no voters:
I20260924 00:59:17.090871 22956 leader_election.cc:288] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [CANDIDATE]: Term 2 election: Requested vote from peers
I20260924 00:59:17.091001 22960 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [term 2 FOLLOWER]: Leader election won for term 2
I20260924 00:59:17.092324 22960 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [term 2 LEADER]: Becoming Leader. State: Replica: 5f62630d137240a3b346349af92107ac, State: Running, Role: LEADER
I20260924 00:59:17.092950 22960 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f62630d137240a3b346349af92107ac" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45505 } }
I20260924 00:59:17.093293 22956 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:59:17.095667 22962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f62630d137240a3b346349af92107ac. Latest consensus state: current_term: 2 leader_uuid: "5f62630d137240a3b346349af92107ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f62630d137240a3b346349af92107ac" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45505 } } }
I20260924 00:59:17.096057 22962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [sys.catalog]: This master's current role is: LEADER
I20260924 00:59:17.096585 22961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5f62630d137240a3b346349af92107ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f62630d137240a3b346349af92107ac" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45505 } } }
I20260924 00:59:17.096869 22961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [sys.catalog]: This master's current role is: LEADER
I20260924 00:59:17.106189 22967 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:59:17.117734 22967 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:59:17.119864 22967 catalog_manager.cc:1259] Loaded cluster ID: d5f43b24f410479c892dd4e5c00bfd87
I20260924 00:59:17.120048 22967 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:59:17.123189 22967 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260924 00:59:17.140250 22967 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260924 00:59:17.150760 22967 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:59:17.152474 22967 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac: Loaded TSK: 0
I20260924 00:59:17.154048 22967 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260924 00:59:17.388561 22958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:59:17.388887 22958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:59:17.389051 22958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:59:17.396670 22958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:59:17.396826 22958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:59:17.396996 22958 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.129
I20260924 00:59:17.406977 22958 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:38521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab
--principal=kudu/127.8.64.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.129:40201
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.64.129
--webserver_port=33331
--tserver_master_addrs=127.8.64.190:45505
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.129
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:59:17.408308 22958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:59:17.409636 22958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:59:17.422441 22984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:59:17.422523 22983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:59:17.423254 22986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:59:17.423745 22958 server_base.cc:1034] running on GCE node
I20260924 00:59:17.424316 22958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:59:17.425267 22958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:59:17.426515 22958 hybrid_clock.cc:648] HybridClock initialized: now 1790211557426447 us; error 55 us; skew 500 ppm
I20260924 00:59:17.426864 22958 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:59:17 dist-test-slave-mmhs krb5kdc[22333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211557, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:59:17.433918 22958 init.cc:377] Logged in from keytab as kudu/127.8.64.129@KRBTEST.COM (short username kudu)
I20260924 00:59:17.435660 22958 webserver.cc:460] Webserver started at http://127.8.64.129:33331/ using document root <none> and password file <none>
I20260924 00:59:17.436565 22958 fs_manager.cc:359] Metadata directory not provided
I20260924 00:59:17.436715 22958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:59:17.443331 22958 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260924 00:59:17.446382 22993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:17.447878 22958 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260924 00:59:17.448046 22958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-0/wal
uuid: "471e118cd0fc492f936acf4f6258b095"
format_stamp: "Formatted at 2026-09-24 00:59:14 on dist-test-slave-mmhs"
server_key: "3eb2520249b6bc14e9ff1a51cd2887fc"
server_key_iv: "a7279037f157517a57403d81aca1653d"
server_key_version: "encryptionkey@0"
I20260924 00:59:17.449322 22958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:59:17.470723 22958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:59:17.475631 22958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:59:17.475934 22958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:59:17.477029 22958 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:59:17.478595 22958 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:59:17.478706 22958 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:17.478801 22958 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:59:17.478873 22958 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:17.512895 22958 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.129:40201
I20260924 00:59:17.512913 23105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.129:40201 every 8 connection(s)
I20260924 00:59:17.514515 22958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
I20260924 00:59:17.523789 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 22958
I20260924 00:59:17.524564 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.130:40057
--local_ip_for_outbound_sockets=127.8.64.130
--tserver_master_addrs=127.8.64.190:45505
--webserver_port=41043
--webserver_interface=127.8.64.130
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab
--principal=kudu/127.8.64.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:38521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 00:59:17 dist-test-slave-mmhs krb5kdc[22333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211557, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:59:17.539340 23108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:17.517326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:45575 (local address 127.8.64.190:45505)
0924 00:59:17.518029 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:17.518046 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:17.518577 (+ 531us) server_negotiation.cc:407] Connection header received
0924 00:59:17.520658 (+ 2081us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:17.520670 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:17.520985 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:17.521327 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:17.522937 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:17.524879 (+ 1942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:17.526482 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:17.527602 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:17.527810 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:17.531338 (+ 3528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:17.531351 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:17.531372 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:17.531429 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:17.534724 (+ 3295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:17.535672 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:17.535680 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:17.535685 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:17.535862 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:17.536400 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:17.536404 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:17.536406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:17.536900 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:17.537053 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:17.537973 (+ 920us) server_negotiation.cc:299] Negotiation successful
0924 00:59:17.538358 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":301,"threads_started":1}
I20260924 00:59:17.539776 23109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:17.517523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.129:45575)
0924 00:59:17.518407 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:17.518447 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 00:59:17.520111 (+ 1664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:59:17.521483 (+ 1372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:59:17.521492 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:59:17.522091 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:59:17.522709 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:17.522726 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:17.525043 (+ 2317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:17.525047 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:17.526278 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:17.526342 (+ 64us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:17.527819 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:17.527823 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:17.528144 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:17.528849 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:59:17.528870 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:59:17.531033 (+ 2163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:59:17.534876 (+ 3843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:59:17.534887 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:59:17.534905 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:59:17.535538 (+ 633us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:59:17.535986 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:59:17.535994 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:59:17.536001 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:59:17.536270 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:59:17.537050 (+ 780us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:59:17.537061 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:59:17.537493 (+ 432us) client_negotiation.cc:769] Sending connection context
0924 00:59:17.537785 (+ 292us) client_negotiation.cc:240] Negotiation successful
0924 00:59:17.538412 (+ 627us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":321,"threads_started":1}
I20260924 00:59:17.542838 23106 heartbeater.cc:347] Connected to a master server at 127.8.64.190:45505
I20260924 00:59:17.543154 23106 heartbeater.cc:464] Registering TS with master...
I20260924 00:59:17.543910 23106 heartbeater.cc:511] Master 127.8.64.190:45505 requested a full tablet report, sending...
I20260924 00:59:17.546439 22921 ts_manager.cc:194] Registered new tserver with Master: 471e118cd0fc492f936acf4f6258b095 (127.8.64.129:40201)
I20260924 00:59:17.548874 22921 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.129@KRBTEST.COM'} at 127.8.64.129:45575
W20260924 00:59:17.856670 23110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:59:17.856935 23110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:59:17.857028 23110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:59:17.864038 23110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:59:17.864143 23110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:59:17.864250 23110 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.130
I20260924 00:59:17.874473 23110 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:38521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab
--principal=kudu/127.8.64.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.130:40057
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.64.130
--webserver_port=41043
--tserver_master_addrs=127.8.64.190:45505
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.130
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:59:17.875682 23110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:59:17.876978 23110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:59:17.889658 23115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:59:17.889882 23116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:59:17.889868 23118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:59:17.890929 23110 server_base.cc:1034] running on GCE node
I20260924 00:59:17.891464 23110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:59:17.892309 23110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:59:17.893522 23110 hybrid_clock.cc:648] HybridClock initialized: now 1790211557893480 us; error 61 us; skew 500 ppm
I20260924 00:59:17.893839 23110 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:59:17 dist-test-slave-mmhs krb5kdc[22333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211557, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:59:17.896895 23110 init.cc:377] Logged in from keytab as kudu/127.8.64.130@KRBTEST.COM (short username kudu)
I20260924 00:59:17.898176 23110 webserver.cc:460] Webserver started at http://127.8.64.130:41043/ using document root <none> and password file <none>
I20260924 00:59:17.898844 23110 fs_manager.cc:359] Metadata directory not provided
I20260924 00:59:17.898953 23110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:59:17.904176 23110 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260924 00:59:17.906795 23125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:17.908054 23110 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260924 00:59:17.908180 23110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-1/wal
uuid: "a9b7028f039c46259584a9163095688a"
format_stamp: "Formatted at 2026-09-24 00:59:14 on dist-test-slave-mmhs"
server_key: "d243346bce3af47d0d00dd6e2ac8b335"
server_key_iv: "86bcea2c24db8983fa633cc9081cddc7"
server_key_version: "encryptionkey@0"
I20260924 00:59:17.909066 23110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:59:17.927733 23110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:59:17.932272 23110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:59:17.932531 23110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:59:17.933511 23110 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:59:17.934875 23110 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:59:17.934984 23110 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:17.935077 23110 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:59:17.935139 23110 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:17.970443 23110 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.130:40057
I20260924 00:59:17.970479 23237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.130:40057 every 8 connection(s)
I20260924 00:59:17.972079 23110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
I20260924 00:59:17.982091 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 23110
I20260924 00:59:17.982899 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.131:35483
--local_ip_for_outbound_sockets=127.8.64.131
--tserver_master_addrs=127.8.64.190:45505
--webserver_port=38823
--webserver_interface=127.8.64.131
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab
--principal=kudu/127.8.64.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:38521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 00:59:17 dist-test-slave-mmhs krb5kdc[22333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211557, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.130@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:59:17.992815 23108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:17.974592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:57729 (local address 127.8.64.190:45505)
0924 00:59:17.974783 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:17.974787 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:17.975913 (+ 1126us) server_negotiation.cc:407] Connection header received
0924 00:59:17.977725 (+ 1812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:17.977730 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:17.977823 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:17.977933 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:17.979406 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:17.980090 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:17.981630 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:17.982365 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:17.982491 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:17.986032 (+ 3541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:17.986041 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:17.986048 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:17.986094 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:17.989664 (+ 3570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:17.990425 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:17.990433 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:17.990439 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:17.990601 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:17.991117 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:17.991120 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:17.991122 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:17.991388 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:17.991507 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:17.992402 (+ 895us) server_negotiation.cc:299] Negotiation successful
0924 00:59:17.992572 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260924 00:59:17.994364 23240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:17.975060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.130:57729)
0924 00:59:17.975746 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:17.975787 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 00:59:17.977449 (+ 1662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:59:17.978173 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:59:17.978181 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:59:17.978685 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:59:17.979231 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:17.979247 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:17.980247 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:17.980252 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:17.981474 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:17.981485 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:17.982751 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:17.982757 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:17.982969 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:17.983723 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:59:17.983749 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:59:17.985804 (+ 2055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:59:17.989812 (+ 4008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:59:17.989819 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:59:17.989831 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:59:17.990248 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:59:17.990713 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:59:17.990716 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:59:17.990719 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:59:17.990993 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:59:17.991515 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:59:17.991526 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:59:17.992007 (+ 481us) client_negotiation.cc:769] Sending connection context
0924 00:59:17.992315 (+ 308us) client_negotiation.cc:240] Negotiation successful
0924 00:59:17.992893 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":276,"threads_started":1}
I20260924 00:59:17.995899 23238 heartbeater.cc:347] Connected to a master server at 127.8.64.190:45505
I20260924 00:59:17.996228 23238 heartbeater.cc:464] Registering TS with master...
I20260924 00:59:17.996940 23238 heartbeater.cc:511] Master 127.8.64.190:45505 requested a full tablet report, sending...
I20260924 00:59:17.998417 22921 ts_manager.cc:194] Registered new tserver with Master: a9b7028f039c46259584a9163095688a (127.8.64.130:40057)
I20260924 00:59:17.999229 22921 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.130@KRBTEST.COM'} at 127.8.64.130:57729
W20260924 00:59:18.316054 23241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:59:18.316340 23241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:59:18.316432 23241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:59:18.323364 23241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:59:18.323465 23241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:59:18.323577 23241 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.131
I20260924 00:59:18.332862 23241 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:38521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab
--principal=kudu/127.8.64.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.131:35483
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.64.131
--webserver_port=38823
--tserver_master_addrs=127.8.64.190:45505
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.131
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:59:18.334036 23241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:59:18.335296 23241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:59:18.348167 23246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:59:18.348222 23247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:59:18.348248 23249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:59:18.349014 23241 server_base.cc:1034] running on GCE node
I20260924 00:59:18.349550 23241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:59:18.350484 23241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:59:18.351732 23241 hybrid_clock.cc:648] HybridClock initialized: now 1790211558351640 us; error 57 us; skew 500 ppm
I20260924 00:59:18.352085 23241 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:59:18 dist-test-slave-mmhs krb5kdc[22333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211558, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:59:18.355165 23241 init.cc:377] Logged in from keytab as kudu/127.8.64.131@KRBTEST.COM (short username kudu)
I20260924 00:59:18.356616 23241 webserver.cc:460] Webserver started at http://127.8.64.131:38823/ using document root <none> and password file <none>
I20260924 00:59:18.357309 23241 fs_manager.cc:359] Metadata directory not provided
I20260924 00:59:18.357462 23241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:59:18.362761 23241 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260924 00:59:18.365547 23256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:18.367039 23241 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 00:59:18.367234 23241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-2/wal
uuid: "18e00ac71aed4617b1a6078ddc64d228"
format_stamp: "Formatted at 2026-09-24 00:59:15 on dist-test-slave-mmhs"
server_key: "c0a2219cbd355d4f8f49662f8f1e868c"
server_key_iv: "05b8aa2e023b621cf8fe3225a158b6f1"
server_key_version: "encryptionkey@0"
I20260924 00:59:18.368273 23241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:59:18.385852 23241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:59:18.389664 23241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:59:18.389882 23241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:59:18.390852 23241 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:59:18.392043 23241 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:59:18.392129 23241 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:18.392199 23241 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:59:18.392256 23241 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:18.425405 23241 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.131:35483
I20260924 00:59:18.425459 23368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.131:35483 every 8 connection(s)
I20260924 00:59:18.426931 23241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
I20260924 00:59:18.429031 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 23241
Sep 24 00:59:18 dist-test-slave-mmhs krb5kdc[22333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211558, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.131@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:59:18.448732 23373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:18.430234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45040 (local address 127.8.64.190:45505)
0924 00:59:18.433671 (+ 3437us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:18.433675 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:18.433690 (+ 15us) server_negotiation.cc:407] Connection header received
0924 00:59:18.433733 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:18.433736 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:18.433785 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:18.433870 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:18.435715 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:18.436783 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:18.438607 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:18.439331 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:18.439460 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:18.442803 (+ 3343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:18.442807 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:18.442810 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:18.442840 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:18.446019 (+ 3179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:18.446741 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:18.446745 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:18.446748 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:18.446867 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:18.447243 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:18.447247 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:18.447257 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:18.447512 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:18.447776 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:18.448090 (+ 314us) server_negotiation.cc:299] Negotiation successful
0924 00:59:18.448311 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":3347,"thread_start_us":3236,"threads_started":1}
I20260924 00:59:18.449224 23108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:18.429595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:35165 (local address 127.8.64.190:45505)
0924 00:59:18.429827 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:18.429831 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:18.431104 (+ 1273us) server_negotiation.cc:407] Connection header received
0924 00:59:18.432966 (+ 1862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:18.432971 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:18.433063 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:18.433169 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:18.435356 (+ 2187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:18.436482 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:18.438766 (+ 2284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:18.439539 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:18.439656 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:18.443529 (+ 3873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:18.443533 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:18.443536 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:18.443573 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:18.446349 (+ 2776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:18.447035 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:18.447041 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:18.447046 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:18.447163 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:18.447582 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:18.447586 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:18.447590 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:18.447847 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:18.447995 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:18.448752 (+ 757us) server_negotiation.cc:299] Negotiation successful
0924 00:59:18.448946 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":100}
I20260924 00:59:18.450629 23372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:18.430092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.131:35165)
0924 00:59:18.430943 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:18.430989 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0924 00:59:18.432705 (+ 1716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:59:18.433577 (+ 872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:59:18.433591 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:59:18.434345 (+ 754us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:59:18.435164 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:18.435193 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:18.436654 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:18.436658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:18.438656 (+ 1998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:18.438668 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:18.439782 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:18.439790 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:18.440082 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:18.440644 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:59:18.440674 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:59:18.443322 (+ 2648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:59:18.446485 (+ 3163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:59:18.446493 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:59:18.446508 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:59:18.446908 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:59:18.447276 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:59:18.447281 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:59:18.447285 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:59:18.447462 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:59:18.447975 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:59:18.447983 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:59:18.448305 (+ 322us) client_negotiation.cc:769] Sending connection context
0924 00:59:18.448615 (+ 310us) client_negotiation.cc:240] Negotiation successful
0924 00:59:18.449476 (+ 861us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":404,"threads_started":1}
I20260924 00:59:18.451973 23369 heartbeater.cc:347] Connected to a master server at 127.8.64.190:45505
I20260924 00:59:18.452234 23369 heartbeater.cc:464] Registering TS with master...
I20260924 00:59:18.452872 23369 heartbeater.cc:511] Master 127.8.64.190:45505 requested a full tablet report, sending...
I20260924 00:59:18.454175 22921 ts_manager.cc:194] Registered new tserver with Master: 18e00ac71aed4617b1a6078ddc64d228 (127.8.64.131:35483)
I20260924 00:59:18.454994 22921 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.131@KRBTEST.COM'} at 127.8.64.131:35165
I20260924 00:59:18.463464 8450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 00:59:18.469828 23108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:18.464318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45056 (local address 127.8.64.190:45505)
0924 00:59:18.464465 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:18.464470 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:18.464888 (+ 418us) server_negotiation.cc:407] Connection header received
0924 00:59:18.465038 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:18.465042 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:18.465165 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:18.465283 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:59:18.465689 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:18.466509 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:18.467930 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:18.468314 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:18.468400 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:18.468834 (+ 434us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:59:18.469295 (+ 461us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:59:18.469400 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:18.469590 (+ 190us) server_negotiation.cc:299] Negotiation successful
0924 00:59:18.469653 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260924 00:59:18.470703 22921 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:45056:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:59:18.477602 22921 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:59:18.511751 23383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:18.489860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41880 (local address 127.8.64.129:40201)
0924 00:59:18.502191 (+ 12331us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:18.502204 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:18.502245 (+ 41us) server_negotiation.cc:407] Connection header received
0924 00:59:18.502415 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:18.502429 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:18.502727 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:18.502937 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:59:18.504167 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:18.505741 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:18.508737 (+ 2996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:18.510227 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:18.510434 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:18.510664 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:18.511257 (+ 593us) server_negotiation.cc:299] Negotiation successful
0924 00:59:18.511419 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12117,"thread_start_us":11925,"threads_started":1}
I20260924 00:59:18.511853 23379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:18.488759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:40201 (local address 127.0.0.1:41880)
0924 00:59:18.501075 (+ 12316us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:18.501099 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0924 00:59:18.501264 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:59:18.503293 (+ 2029us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:59:18.503306 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:59:18.503340 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:59:18.503937 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:18.503958 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:18.505916 (+ 1958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:18.505924 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:18.508534 (+ 2610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:18.508555 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:18.510817 (+ 2262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:18.510822 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:18.510992 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:18.511045 (+ 53us) client_negotiation.cc:769] Sending connection context
0924 00:59:18.511214 (+ 169us) client_negotiation.cc:240] Negotiation successful
0924 00:59:18.511396 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12149,"mutex_wait_us":24,"thread_start_us":13764,"threads_started":1}
I20260924 00:59:18.512353 23382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:18.488580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.131:35483 (local address 127.0.0.1:46968)
0924 00:59:18.497132 (+ 8552us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:18.497186 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0924 00:59:18.497423 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:59:18.504794 (+ 7371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:59:18.504799 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:59:18.504823 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:59:18.505147 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:18.505157 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:18.507384 (+ 2227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:18.507392 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:18.509083 (+ 1691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:18.509097 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:18.511485 (+ 2388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:18.511492 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:18.511670 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:18.511706 (+ 36us) client_negotiation.cc:769] Sending connection context
0924 00:59:18.511809 (+ 103us) client_negotiation.cc:240] Negotiation successful
0924 00:59:18.511945 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8169,"mutex_wait_us":23,"thread_start_us":12784,"threads_started":1}
I20260924 00:59:18.513175 23380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:18.489112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.130:40057 (local address 127.0.0.1:56442)
0924 00:59:18.502889 (+ 13777us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:18.502918 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0924 00:59:18.503056 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:59:18.504008 (+ 952us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:59:18.504013 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:59:18.504041 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:59:18.504416 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:18.504427 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:18.506429 (+ 2002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:18.506435 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:18.508636 (+ 2201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:18.508651 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:18.511860 (+ 3209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:18.512078 (+ 218us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:18.512414 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:18.512452 (+ 38us) client_negotiation.cc:769] Sending connection context
0924 00:59:18.512633 (+ 181us) client_negotiation.cc:240] Negotiation successful
0924 00:59:18.512781 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13583,"mutex_wait_us":24,"thread_start_us":7452,"threads_started":1}
I20260924 00:59:18.513775 23381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:18.488830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46968 (local address 127.8.64.131:35483)
0924 00:59:18.504174 (+ 15344us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:18.504180 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:18.504208 (+ 28us) server_negotiation.cc:407] Connection header received
0924 00:59:18.504287 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:18.504292 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:18.504554 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:18.504782 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:59:18.505344 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:18.507255 (+ 1911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:18.509245 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:18.510981 (+ 1736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:18.511107 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:18.511314 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:18.512990 (+ 1676us) server_negotiation.cc:299] Negotiation successful
0924 00:59:18.513161 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15154,"thread_start_us":14968,"threads_started":1}
I20260924 00:59:18.514092 23384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:18.490446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56442 (local address 127.8.64.130:40057)
0924 00:59:18.497526 (+ 7080us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:18.497534 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:18.503013 (+ 5479us) server_negotiation.cc:407] Connection header received
0924 00:59:18.503314 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:18.503327 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:18.503595 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:18.503814 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:59:18.504608 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:18.506312 (+ 1704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:18.508799 (+ 2487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:18.511316 (+ 2517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:18.511527 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:18.511702 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:18.513543 (+ 1841us) server_negotiation.cc:299] Negotiation successful
0924 00:59:18.513799 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6945,"thread_start_us":6874,"threads_started":1}
I20260924 00:59:18.516460 23041 tablet_service.cc:1467] Processing CreateTablet for tablet e5cad1078738455e96ac01f143f734ef (DEFAULT_TABLE table=test-table [id=fca7094af3ea45258f97b34a5c1908c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:59:18.517328 23304 tablet_service.cc:1467] Processing CreateTablet for tablet e5cad1078738455e96ac01f143f734ef (DEFAULT_TABLE table=test-table [id=fca7094af3ea45258f97b34a5c1908c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:59:18.517328 23173 tablet_service.cc:1467] Processing CreateTablet for tablet e5cad1078738455e96ac01f143f734ef (DEFAULT_TABLE table=test-table [id=fca7094af3ea45258f97b34a5c1908c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:59:18.517921 23041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5cad1078738455e96ac01f143f734ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:59:18.518581 23304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5cad1078738455e96ac01f143f734ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:59:18.518584 23173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5cad1078738455e96ac01f143f734ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:59:18.530176 23385 tablet_bootstrap.cc:492] T e5cad1078738455e96ac01f143f734ef P 471e118cd0fc492f936acf4f6258b095: Bootstrap starting.
I20260924 00:59:18.533213 23385 tablet_bootstrap.cc:654] T e5cad1078738455e96ac01f143f734ef P 471e118cd0fc492f936acf4f6258b095: Neither blocks nor log segments found. Creating new log.
I20260924 00:59:18.534590 23385 log.cc:824] T e5cad1078738455e96ac01f143f734ef P 471e118cd0fc492f936acf4f6258b095: Log is configured to *not* fsync() on all Append() calls
I20260924 00:59:18.534813 23387 tablet_bootstrap.cc:492] T e5cad1078738455e96ac01f143f734ef P 18e00ac71aed4617b1a6078ddc64d228: Bootstrap starting.
I20260924 00:59:18.539013 23386 tablet_bootstrap.cc:492] T e5cad1078738455e96ac01f143f734ef P a9b7028f039c46259584a9163095688a: Bootstrap starting.
I20260924 00:59:18.539314 23387 tablet_bootstrap.cc:654] T e5cad1078738455e96ac01f143f734ef P 18e00ac71aed4617b1a6078ddc64d228: Neither blocks nor log segments found. Creating new log.
I20260924 00:59:18.540915 23387 log.cc:824] T e5cad1078738455e96ac01f143f734ef P 18e00ac71aed4617b1a6078ddc64d228: Log is configured to *not* fsync() on all Append() calls
I20260924 00:59:18.542750 23386 tablet_bootstrap.cc:654] T e5cad1078738455e96ac01f143f734ef P a9b7028f039c46259584a9163095688a: Neither blocks nor log segments found. Creating new log.
I20260924 00:59:18.543784 23385 tablet_bootstrap.cc:492] T e5cad1078738455e96ac01f143f734ef P 471e118cd0fc492f936acf4f6258b095: No bootstrap required, opened a new log
I20260924 00:59:18.544003 23386 log.cc:824] T e5cad1078738455e96ac01f143f734ef P a9b7028f039c46259584a9163095688a: Log is configured to *not* fsync() on all Append() calls
I20260924 00:59:18.544072 23385 ts_tablet_manager.cc:1397] T e5cad1078738455e96ac01f143f734ef P 471e118cd0fc492f936acf4f6258b095: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.008s
I20260924 00:59:18.544677 23387 tablet_bootstrap.cc:492] T e5cad1078738455e96ac01f143f734ef P 18e00ac71aed4617b1a6078ddc64d228: No bootstrap required, opened a new log
I20260924 00:59:18.544934 23387 ts_tablet_manager.cc:1397] T e5cad1078738455e96ac01f143f734ef P 18e00ac71aed4617b1a6078ddc64d228: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260924 00:59:18.546981 23386 tablet_bootstrap.cc:492] T e5cad1078738455e96ac01f143f734ef P a9b7028f039c46259584a9163095688a: No bootstrap required, opened a new log
I20260924 00:59:18.547219 23386 ts_tablet_manager.cc:1397] T e5cad1078738455e96ac01f143f734ef P a9b7028f039c46259584a9163095688a: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.001s
I20260924 00:59:18.552611 23385 raft_consensus.cc:348] T e5cad1078738455e96ac01f143f734ef P 471e118cd0fc492f936acf4f6258b095 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18e00ac71aed4617b1a6078ddc64d228" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35483 } } peers { permanent_uuid: "471e118cd0fc492f936acf4f6258b095" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 40201 } } peers { permanent_uuid: "a9b7028f039c46259584a9163095688a" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 40057 } }
I20260924 00:59:18.553006 23385 raft_consensus.cc:374] T e5cad1078738455e96ac01f143f734ef P 471e118cd0fc492f936acf4f6258b095 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:59:18.553126 23385 raft_consensus.cc:729] T e5cad1078738455e96ac01f143f734ef P 471e118cd0fc492f936acf4f6258b095 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 471e118cd0fc492f936acf4f6258b095, State: Initialized, Role: FOLLOWER
I20260924 00:59:18.553270 23106 heartbeater.cc:503] Master 127.8.64.190:45505 was elected leader, sending a full tablet report...
I20260924 00:59:18.553855 23385 consensus_queue.cc:260] T e5cad1078738455e96ac01f143f734ef P 471e118cd0fc492f936acf4f6258b095 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18e00ac71aed4617b1a6078ddc64d228" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35483 } } peers { permanent_uuid: "471e118cd0fc492f936acf4f6258b095" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 40201 } } peers { permanent_uuid: "a9b7028f039c46259584a9163095688a" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 40057 } }
I20260924 00:59:18.557081 23387 raft_consensus.cc:348] T e5cad1078738455e96ac01f143f734ef P 18e00ac71aed4617b1a6078ddc64d228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18e00ac71aed4617b1a6078ddc64d228" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35483 } } peers { permanent_uuid: "471e118cd0fc492f936acf4f6258b095" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 40201 } } peers { permanent_uuid: "a9b7028f039c46259584a9163095688a" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 40057 } }
I20260924 00:59:18.557528 23387 raft_consensus.cc:374] T e5cad1078738455e96ac01f143f734ef P 18e00ac71aed4617b1a6078ddc64d228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:59:18.557616 23387 raft_consensus.cc:729] T e5cad1078738455e96ac01f143f734ef P 18e00ac71aed4617b1a6078ddc64d228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18e00ac71aed4617b1a6078ddc64d228, State: Initialized, Role: FOLLOWER
I20260924 00:59:18.558395 23387 consensus_queue.cc:260] T e5cad1078738455e96ac01f143f734ef P 18e00ac71aed4617b1a6078ddc64d228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18e00ac71aed4617b1a6078ddc64d228" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35483 } } peers { permanent_uuid: "471e118cd0fc492f936acf4f6258b095" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 40201 } } peers { permanent_uuid: "a9b7028f039c46259584a9163095688a" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 40057 } }
I20260924 00:59:18.558434 23386 raft_consensus.cc:348] T e5cad1078738455e96ac01f143f734ef P a9b7028f039c46259584a9163095688a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18e00ac71aed4617b1a6078ddc64d228" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35483 } } peers { permanent_uuid: "471e118cd0fc492f936acf4f6258b095" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 40201 } } peers { permanent_uuid: "a9b7028f039c46259584a9163095688a" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 40057 } }
I20260924 00:59:18.558840 23386 raft_consensus.cc:374] T e5cad1078738455e96ac01f143f734ef P a9b7028f039c46259584a9163095688a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:59:18.558959 23386 raft_consensus.cc:729] T e5cad1078738455e96ac01f143f734ef P a9b7028f039c46259584a9163095688a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9b7028f039c46259584a9163095688a, State: Initialized, Role: FOLLOWER
I20260924 00:59:18.559718 23386 consensus_queue.cc:260] T e5cad1078738455e96ac01f143f734ef P a9b7028f039c46259584a9163095688a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18e00ac71aed4617b1a6078ddc64d228" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35483 } } peers { permanent_uuid: "471e118cd0fc492f936acf4f6258b095" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 40201 } } peers { permanent_uuid: "a9b7028f039c46259584a9163095688a" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 40057 } }
I20260924 00:59:18.562701 23386 ts_tablet_manager.cc:1428] T e5cad1078738455e96ac01f143f734ef P a9b7028f039c46259584a9163095688a: Time spent starting tablet: real 0.015s user 0.012s sys 0.003s
I20260924 00:59:18.562969 23385 ts_tablet_manager.cc:1428] T e5cad1078738455e96ac01f143f734ef P 471e118cd0fc492f936acf4f6258b095: Time spent starting tablet: real 0.019s user 0.009s sys 0.010s
I20260924 00:59:18.563795 23369 heartbeater.cc:503] Master 127.8.64.190:45505 was elected leader, sending a full tablet report...
I20260924 00:59:18.563910 23238 heartbeater.cc:503] Master 127.8.64.190:45505 was elected leader, sending a full tablet report...
I20260924 00:59:18.564405 23387 ts_tablet_manager.cc:1428] T e5cad1078738455e96ac01f143f734ef P 18e00ac71aed4617b1a6078ddc64d228: Time spent starting tablet: real 0.019s user 0.011s sys 0.007s
I20260924 00:59:18.665210 23393 raft_consensus.cc:482] T e5cad1078738455e96ac01f143f734ef P a9b7028f039c46259584a9163095688a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:59:18.665561 23393 raft_consensus.cc:504] T e5cad1078738455e96ac01f143f734ef P a9b7028f039c46259584a9163095688a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18e00ac71aed4617b1a6078ddc64d228" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35483 } } peers { permanent_uuid: "471e118cd0fc492f936acf4f6258b095" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 40201 } } peers { permanent_uuid: "a9b7028f039c46259584a9163095688a" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 40057 } }
I20260924 00:59:18.667395 23393 leader_election.cc:288] T e5cad1078738455e96ac01f143f734ef P a9b7028f039c46259584a9163095688a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18e00ac71aed4617b1a6078ddc64d228 (127.8.64.131:35483), 471e118cd0fc492f936acf4f6258b095 (127.8.64.129:40201)
I20260924 00:59:18.674633 23381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:18.667777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:38587 (local address 127.8.64.131:35483)
0924 00:59:18.667994 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:18.668003 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:18.668886 (+ 883us) server_negotiation.cc:407] Connection header received
0924 00:59:18.669068 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:18.669072 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:18.669130 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:18.669244 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:59:18.669931 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:18.670697 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:18.672703 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:18.673601 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:18.673727 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:18.673783 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:18.674214 (+ 431us) server_negotiation.cc:299] Negotiation successful
0924 00:59:18.674365 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260924 00:59:18.674669 23383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:18.668024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:50543 (local address 127.8.64.129:40201)
0924 00:59:18.668274 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:18.668280 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:18.668815 (+ 535us) server_negotiation.cc:407] Connection header received
0924 00:59:18.669027 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:18.669033 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:18.669105 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:18.669171 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:59:18.669862 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:18.670679 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:18.672891 (+ 2212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:18.673724 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:18.673890 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:18.673969 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:18.674263 (+ 294us) server_negotiation.cc:299] Negotiation successful
0924 00:59:18.674412 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260924 00:59:18.674957 23395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:18.667774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.131:35483 (local address 127.8.64.130:38587)
0924 00:59:18.668735 (+ 961us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:18.668749 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 00:59:18.668907 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:59:18.669281 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:59:18.669286 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:59:18.669315 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:59:18.669751 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:18.669759 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:18.670855 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:18.670862 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:18.672549 (+ 1687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:18.672561 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:18.673850 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:18.673856 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:18.674050 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:18.674074 (+ 24us) client_negotiation.cc:769] Sending connection context
0924 00:59:18.674154 (+ 80us) client_negotiation.cc:240] Negotiation successful
0924 00:59:18.674334 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":845,"mutex_wait_us":1,"thread_start_us":674,"threads_started":1}
I20260924 00:59:18.675067 23394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:18.667650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:40201 (local address 127.8.64.130:50543)
0924 00:59:18.668674 (+ 1024us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:18.668702 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0924 00:59:18.668867 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:59:18.669221 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:59:18.669229 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:59:18.669252 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:59:18.669715 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:18.669725 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:18.670808 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:18.670813 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:18.672713 (+ 1900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:18.672727 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:18.673948 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:18.673953 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:18.674071 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:18.674092 (+ 21us) client_negotiation.cc:769] Sending connection context
0924 00:59:18.674208 (+ 116us) client_negotiation.cc:240] Negotiation successful
0924 00:59:18.674363 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":873,"mutex_wait_us":209,"thread_start_us":598,"threads_started":1}
I20260924 00:59:18.675962 23324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5cad1078738455e96ac01f143f734ef" candidate_uuid: "a9b7028f039c46259584a9163095688a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18e00ac71aed4617b1a6078ddc64d228" is_pre_election: true
I20260924 00:59:18.676012 23061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5cad1078738455e96ac01f143f734ef" candidate_uuid: "a9b7028f039c46259584a9163095688a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "471e118cd0fc492f936acf4f6258b095" is_pre_election: true
I20260924 00:59:18.676460 23324 raft_consensus.cc:2454] T e5cad1078738455e96ac01f143f734ef P 18e00ac71aed4617b1a6078ddc64d228 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9b7028f039c46259584a9163095688a in term 0.
I20260924 00:59:18.676540 23061 raft_consensus.cc:2454] T e5cad1078738455e96ac01f143f734ef P 471e118cd0fc492f936acf4f6258b095 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9b7028f039c46259584a9163095688a in term 0.
I20260924 00:59:18.677202 23128 leader_election.cc:302] T e5cad1078738455e96ac01f143f734ef P a9b7028f039c46259584a9163095688a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18e00ac71aed4617b1a6078ddc64d228, a9b7028f039c46259584a9163095688a; no voters:
I20260924 00:59:18.677597 23393 raft_consensus.cc:2781] T e5cad1078738455e96ac01f143f734ef P a9b7028f039c46259584a9163095688a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:59:18.677762 23393 raft_consensus.cc:482] T e5cad1078738455e96ac01f143f734ef P a9b7028f039c46259584a9163095688a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:59:18.677875 23393 raft_consensus.cc:3037] T e5cad1078738455e96ac01f143f734ef P a9b7028f039c46259584a9163095688a [term 0 FOLLOWER]: Advancing to term 1
W20260924 00:59:18.678469 23370 tablet.cc:2369] T e5cad1078738455e96ac01f143f734ef P 18e00ac71aed4617b1a6078ddc64d228: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:59:18.680423 23393 raft_consensus.cc:504] T e5cad1078738455e96ac01f143f734ef P a9b7028f039c46259584a9163095688a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18e00ac71aed4617b1a6078ddc64d228" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35483 } } peers { permanent_uuid: "471e118cd0fc492f936acf4f6258b095" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 40201 } } peers { permanent_uuid: "a9b7028f039c46259584a9163095688a" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 40057 } }
I20260924 00:59:18.681039 23393 leader_election.cc:288] T e5cad1078738455e96ac01f143f734ef P a9b7028f039c46259584a9163095688a [CANDIDATE]: Term 1 election: Requested vote from peers 18e00ac71aed4617b1a6078ddc64d228 (127.8.64.131:35483), 471e118cd0fc492f936acf4f6258b095 (127.8.64.129:40201)
I20260924 00:59:18.681442 23324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5cad1078738455e96ac01f143f734ef" candidate_uuid: "a9b7028f039c46259584a9163095688a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18e00ac71aed4617b1a6078ddc64d228"
I20260924 00:59:18.681548 23061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5cad1078738455e96ac01f143f734ef" candidate_uuid: "a9b7028f039c46259584a9163095688a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "471e118cd0fc492f936acf4f6258b095"
I20260924 00:59:18.681704 23324 raft_consensus.cc:3037] T e5cad1078738455e96ac01f143f734ef P 18e00ac71aed4617b1a6078ddc64d228 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:59:18.681814 23061 raft_consensus.cc:3037] T e5cad1078738455e96ac01f143f734ef P 471e118cd0fc492f936acf4f6258b095 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:59:18.684125 23324 raft_consensus.cc:2454] T e5cad1078738455e96ac01f143f734ef P 18e00ac71aed4617b1a6078ddc64d228 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9b7028f039c46259584a9163095688a in term 1.
I20260924 00:59:18.684181 23061 raft_consensus.cc:2454] T e5cad1078738455e96ac01f143f734ef P 471e118cd0fc492f936acf4f6258b095 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9b7028f039c46259584a9163095688a in term 1.
I20260924 00:59:18.684556 23128 leader_election.cc:302] T e5cad1078738455e96ac01f143f734ef P a9b7028f039c46259584a9163095688a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18e00ac71aed4617b1a6078ddc64d228, a9b7028f039c46259584a9163095688a; no voters:
I20260924 00:59:18.684844 23393 raft_consensus.cc:2781] T e5cad1078738455e96ac01f143f734ef P a9b7028f039c46259584a9163095688a [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:59:18.685354 23393 raft_consensus.cc:686] T e5cad1078738455e96ac01f143f734ef P a9b7028f039c46259584a9163095688a [term 1 LEADER]: Becoming Leader. State: Replica: a9b7028f039c46259584a9163095688a, State: Running, Role: LEADER
I20260924 00:59:18.685845 23393 consensus_queue.cc:237] T e5cad1078738455e96ac01f143f734ef P a9b7028f039c46259584a9163095688a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18e00ac71aed4617b1a6078ddc64d228" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35483 } } peers { permanent_uuid: "471e118cd0fc492f936acf4f6258b095" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 40201 } } peers { permanent_uuid: "a9b7028f039c46259584a9163095688a" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 40057 } }
I20260924 00:59:18.690191 22920 catalog_manager.cc:5495] T e5cad1078738455e96ac01f143f734ef P a9b7028f039c46259584a9163095688a reported cstate change: term changed from 0 to 1, leader changed from <none> to a9b7028f039c46259584a9163095688a (127.8.64.130). New cstate: current_term: 1 leader_uuid: "a9b7028f039c46259584a9163095688a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18e00ac71aed4617b1a6078ddc64d228" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "471e118cd0fc492f936acf4f6258b095" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 40201 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9b7028f039c46259584a9163095688a" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 40057 } health_report { overall_health: HEALTHY } } }
I20260924 00:59:18.712365 23384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:18.707057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56452 (local address 127.8.64.130:40057)
0924 00:59:18.707302 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:18.707309 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:18.707328 (+ 19us) server_negotiation.cc:407] Connection header received
0924 00:59:18.707547 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:18.707557 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:18.707633 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:18.707778 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:59:18.708137 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:18.708826 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:18.710188 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:18.710662 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:18.710788 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:18.711284 (+ 496us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:59:18.711759 (+ 475us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:59:18.711924 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:18.711974 (+ 50us) server_negotiation.cc:299] Negotiation successful
0924 00:59:18.712051 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260924 00:59:18.720721 23324 raft_consensus.cc:1261] T e5cad1078738455e96ac01f143f734ef P 18e00ac71aed4617b1a6078ddc64d228 [term 1 FOLLOWER]: Refusing update from remote peer a9b7028f039c46259584a9163095688a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:59:18.720767 23061 raft_consensus.cc:1261] T e5cad1078738455e96ac01f143f734ef P 471e118cd0fc492f936acf4f6258b095 [term 1 FOLLOWER]: Refusing update from remote peer a9b7028f039c46259584a9163095688a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:59:18.721913 23396 consensus_queue.cc:1035] T e5cad1078738455e96ac01f143f734ef P a9b7028f039c46259584a9163095688a [LEADER]: Connected to new peer: Peer: permanent_uuid: "18e00ac71aed4617b1a6078ddc64d228" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 35483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:59:18.722337 23393 consensus_queue.cc:1035] T e5cad1078738455e96ac01f143f734ef P a9b7028f039c46259584a9163095688a [LEADER]: Connected to new peer: Peer: permanent_uuid: "471e118cd0fc492f936acf4f6258b095" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 40201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260924 00:59:18.724051 23239 tablet.cc:2369] T e5cad1078738455e96ac01f143f734ef P a9b7028f039c46259584a9163095688a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:59:18.733103 23398 mvcc.cc:205] Tried to move back new op lower bound from 7332706544509194240 to 7332706544381403136. Current Snapshot: MvccSnapshot[applied={T|T < 7332706544509194240}]
I20260924 00:59:18.748499 22919 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:45056:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:59:18.748799 22919 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:45056:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:59:18.752435 22919 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac: Sending DeleteTablet for 3 replicas of tablet e5cad1078738455e96ac01f143f734ef
I20260924 00:59:18.753685 23304 tablet_service.cc:1514] Processing DeleteTablet for tablet e5cad1078738455e96ac01f143f734ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:59:18 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:46968
I20260924 00:59:18.753787 23041 tablet_service.cc:1514] Processing DeleteTablet for tablet e5cad1078738455e96ac01f143f734ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:59:18 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:41880
I20260924 00:59:18.753993 23173 tablet_service.cc:1514] Processing DeleteTablet for tablet e5cad1078738455e96ac01f143f734ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:59:18 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:56442
I20260924 00:59:18.754949 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 22958
I20260924 00:59:18.764014 23411 tablet_replica.cc:330] T e5cad1078738455e96ac01f143f734ef P 471e118cd0fc492f936acf4f6258b095: stopping tablet replica
I20260924 00:59:18.764732 23411 raft_consensus.cc:2229] T e5cad1078738455e96ac01f143f734ef P 471e118cd0fc492f936acf4f6258b095 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:59:18.765427 23411 raft_consensus.cc:2258] T e5cad1078738455e96ac01f143f734ef P 471e118cd0fc492f936acf4f6258b095 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:59:18.766501 23412 tablet_replica.cc:330] T e5cad1078738455e96ac01f143f734ef P a9b7028f039c46259584a9163095688a: stopping tablet replica
I20260924 00:59:18.766938 23412 raft_consensus.cc:2229] T e5cad1078738455e96ac01f143f734ef P a9b7028f039c46259584a9163095688a [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:59:18.767621 23412 raft_consensus.cc:2258] T e5cad1078738455e96ac01f143f734ef P a9b7028f039c46259584a9163095688a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:59:18.767908 23411 ts_tablet_manager.cc:1905] T e5cad1078738455e96ac01f143f734ef P 471e118cd0fc492f936acf4f6258b095: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:59:18.769685 23412 ts_tablet_manager.cc:1905] T e5cad1078738455e96ac01f143f734ef P a9b7028f039c46259584a9163095688a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:59:18.770249 23409 tablet_replica.cc:330] T e5cad1078738455e96ac01f143f734ef P 18e00ac71aed4617b1a6078ddc64d228: stopping tablet replica
I20260924 00:59:18.772045 23409 raft_consensus.cc:2229] T e5cad1078738455e96ac01f143f734ef P 18e00ac71aed4617b1a6078ddc64d228 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:59:18.772653 23409 raft_consensus.cc:2258] T e5cad1078738455e96ac01f143f734ef P 18e00ac71aed4617b1a6078ddc64d228 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:59:18.777560 23411 ts_tablet_manager.cc:1918] T e5cad1078738455e96ac01f143f734ef P 471e118cd0fc492f936acf4f6258b095: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:59:18.777697 23412 ts_tablet_manager.cc:1918] T e5cad1078738455e96ac01f143f734ef P a9b7028f039c46259584a9163095688a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:59:18.777880 23411 log.cc:1196] T e5cad1078738455e96ac01f143f734ef P 471e118cd0fc492f936acf4f6258b095: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-0/wal/wals/e5cad1078738455e96ac01f143f734ef
I20260924 00:59:18.777885 23412 log.cc:1196] T e5cad1078738455e96ac01f143f734ef P a9b7028f039c46259584a9163095688a: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-1/wal/wals/e5cad1078738455e96ac01f143f734ef
I20260924 00:59:18.778507 23411 ts_tablet_manager.cc:1939] T e5cad1078738455e96ac01f143f734ef P 471e118cd0fc492f936acf4f6258b095: Deleting consensus metadata
I20260924 00:59:18.778529 23412 ts_tablet_manager.cc:1939] T e5cad1078738455e96ac01f143f734ef P a9b7028f039c46259584a9163095688a: Deleting consensus metadata
I20260924 00:59:18.779717 23409 ts_tablet_manager.cc:1905] T e5cad1078738455e96ac01f143f734ef P 18e00ac71aed4617b1a6078ddc64d228: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:59:18.780319 22904 catalog_manager.cc:4841] TS 471e118cd0fc492f936acf4f6258b095 (127.8.64.129:40201): tablet e5cad1078738455e96ac01f143f734ef (table test-table [id=fca7094af3ea45258f97b34a5c1908c5]) successfully deleted
I20260924 00:59:18.780447 22907 catalog_manager.cc:4841] TS a9b7028f039c46259584a9163095688a (127.8.64.130:40057): tablet e5cad1078738455e96ac01f143f734ef (table test-table [id=fca7094af3ea45258f97b34a5c1908c5]) successfully deleted
I20260924 00:59:18.781292 23383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:18.756394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41888 (local address 127.8.64.129:40201)
0924 00:59:18.756564 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:18.756569 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:18.756588 (+ 19us) server_negotiation.cc:407] Connection header received
0924 00:59:18.756653 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:18.756660 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:18.756751 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:18.756850 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:18.758980 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:18.760203 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:18.762586 (+ 2383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:18.763265 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:18.763389 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:18.770147 (+ 6758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:18.770162 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:18.770178 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:18.770235 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:18.774454 (+ 4219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:18.775411 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:18.775421 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:18.775425 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:18.775597 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:18.776093 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:18.776098 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:18.776102 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:18.776641 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:18.776950 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:18.780875 (+ 3925us) server_negotiation.cc:299] Negotiation successful
0924 00:59:18.781059 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260924 00:59:18.782430 23101 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:59:18.787593 23409 ts_tablet_manager.cc:1918] T e5cad1078738455e96ac01f143f734ef P 18e00ac71aed4617b1a6078ddc64d228: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:59:18.787760 23409 log.cc:1196] T e5cad1078738455e96ac01f143f734ef P 18e00ac71aed4617b1a6078ddc64d228: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-2/wal/wals/e5cad1078738455e96ac01f143f734ef
I20260924 00:59:18.788302 23409 ts_tablet_manager.cc:1939] T e5cad1078738455e96ac01f143f734ef P 18e00ac71aed4617b1a6078ddc64d228: Deleting consensus metadata
I20260924 00:59:18.790198 22906 catalog_manager.cc:4841] TS 18e00ac71aed4617b1a6078ddc64d228 (127.8.64.131:35483): tablet e5cad1078738455e96ac01f143f734ef (table test-table [id=fca7094af3ea45258f97b34a5c1908c5]) successfully deleted
I20260924 00:59:18.871596 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 22958
I20260924 00:59:18.897184 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 23110
I20260924 00:59:18.912401 23384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:18.898002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56460 (local address 127.8.64.130:40057)
0924 00:59:18.898258 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:18.898264 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:18.898283 (+ 19us) server_negotiation.cc:407] Connection header received
0924 00:59:18.898384 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:18.898391 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:18.898469 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:18.898590 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:18.899998 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:18.900871 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:18.902167 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:18.902743 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:18.902892 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:18.905678 (+ 2786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:18.905690 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:18.905705 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:18.905748 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:18.909139 (+ 3391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:18.909840 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:18.909849 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:18.909854 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:18.910011 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:18.910454 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:18.910461 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:18.910463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:18.910942 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:18.911257 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:18.911729 (+ 472us) server_negotiation.cc:299] Negotiation successful
0924 00:59:18.911938 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260924 00:59:18.915305 23233 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:59:19.027384 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 23110
I20260924 00:59:19.048684 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 23241
I20260924 00:59:19.061818 23381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:19.049380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46980 (local address 127.8.64.131:35483)
0924 00:59:19.049577 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:19.049582 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:19.049599 (+ 17us) server_negotiation.cc:407] Connection header received
0924 00:59:19.049789 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:19.049793 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:19.049857 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:19.049939 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:19.051273 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:19.051985 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:19.053264 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:19.053688 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:19.053796 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:19.056387 (+ 2591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:19.056401 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:19.056417 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:19.056454 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:19.059346 (+ 2892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:19.060023 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:19.060030 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:19.060032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:19.060172 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:19.060477 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:19.060480 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:19.060482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:19.060820 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:19.061025 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:19.061286 (+ 261us) server_negotiation.cc:299] Negotiation successful
0924 00:59:19.061449 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260924 00:59:19.062541 23364 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:59:19.152164 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 23241
I20260924 00:59:19.169049 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 22887
I20260924 00:59:19.170369 22951 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:59:19.287556 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 22887
I20260924 00:59:19.309412 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:45505
--webserver_interface=127.8.64.190
--webserver_port=33931
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:38521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:45505
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:59:19.634056 23417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:59:19.634366 23417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:59:19.634460 23417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:59:19.641561 23417 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:59:19.641666 23417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:59:19.641712 23417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:59:19.641770 23417 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:59:19.641866 23417 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:59:19.652204 23417 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:38521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:45505
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:45505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=33931
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:59:19.653559 23417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:59:19.654920 23417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:59:19.666966 23425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:59:19.666989 23423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:59:19.667038 23422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:59:19.668152 23417 server_base.cc:1034] running on GCE node
I20260924 00:59:19.668946 23417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:59:19.670441 23417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:59:19.671660 23417 hybrid_clock.cc:648] HybridClock initialized: now 1790211559671607 us; error 58 us; skew 500 ppm
I20260924 00:59:19.672008 23417 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:59:19 dist-test-slave-mmhs krb5kdc[22333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211559, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:59:19.675261 23417 init.cc:377] Logged in from keytab as kudu/127.8.64.190@KRBTEST.COM (short username kudu)
I20260924 00:59:19.676833 23417 webserver.cc:460] Webserver started at http://127.8.64.190:33931/ using document root <none> and password file <none>
I20260924 00:59:19.677546 23417 fs_manager.cc:359] Metadata directory not provided
I20260924 00:59:19.677700 23417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:59:19.683316 23417 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260924 00:59:19.686057 23432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:19.687438 23417 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260924 00:59:19.687619 23417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "5f62630d137240a3b346349af92107ac"
format_stamp: "Formatted at 2026-09-24 00:59:13 on dist-test-slave-mmhs"
server_key: "b54bffb460091fa8d7c93f15610aa6fd"
server_key_iv: "5f9f7dd50abfd48e2f0afa2902d5362d"
server_key_version: "encryptionkey@0"
I20260924 00:59:19.688599 23417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:59:19.704270 23417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:59:19.708616 23417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:59:19.708920 23417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:59:19.728118 23417 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.190:45505
I20260924 00:59:19.728145 23483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.190:45505 every 8 connection(s)
I20260924 00:59:19.729683 23417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 00:59:19.731489 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 23417
I20260924 00:59:19.732384 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.129:40201
--local_ip_for_outbound_sockets=127.8.64.129
--tserver_master_addrs=127.8.64.190:45505
--webserver_port=33331
--webserver_interface=127.8.64.129
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab
--principal=kudu/127.8.64.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:38521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 00:59:19.736932 23484 sys_catalog.cc:263] Verifying existing consensus state
I20260924 00:59:19.740175 23484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac: Bootstrap starting.
I20260924 00:59:19.760988 23484 log.cc:824] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac: Log is configured to *not* fsync() on all Append() calls
I20260924 00:59:19.777097 23484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 00:59:19.777647 23484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac: Bootstrap complete.
I20260924 00:59:19.786816 23484 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f62630d137240a3b346349af92107ac" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45505 } }
I20260924 00:59:19.787730 23484 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f62630d137240a3b346349af92107ac, State: Initialized, Role: FOLLOWER
I20260924 00:59:19.788369 23484 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f62630d137240a3b346349af92107ac" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45505 } }
I20260924 00:59:19.788625 23484 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:59:19.788748 23484 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:59:19.788928 23484 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [term 2 FOLLOWER]: Advancing to term 3
I20260924 00:59:19.792974 23484 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f62630d137240a3b346349af92107ac" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45505 } }
I20260924 00:59:19.793500 23484 leader_election.cc:302] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f62630d137240a3b346349af92107ac; no voters:
I20260924 00:59:19.794188 23484 leader_election.cc:288] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [CANDIDATE]: Term 3 election: Requested vote from peers
I20260924 00:59:19.794333 23488 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [term 3 FOLLOWER]: Leader election won for term 3
I20260924 00:59:19.795595 23488 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [term 3 LEADER]: Becoming Leader. State: Replica: 5f62630d137240a3b346349af92107ac, State: Running, Role: LEADER
I20260924 00:59:19.796181 23488 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f62630d137240a3b346349af92107ac" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45505 } }
I20260924 00:59:19.796515 23484 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:59:19.798727 23490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f62630d137240a3b346349af92107ac. Latest consensus state: current_term: 3 leader_uuid: "5f62630d137240a3b346349af92107ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f62630d137240a3b346349af92107ac" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45505 } } }
I20260924 00:59:19.799093 23490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [sys.catalog]: This master's current role is: LEADER
I20260924 00:59:19.799562 23489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "5f62630d137240a3b346349af92107ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f62630d137240a3b346349af92107ac" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45505 } } }
I20260924 00:59:19.800143 23489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f62630d137240a3b346349af92107ac [sys.catalog]: This master's current role is: LEADER
I20260924 00:59:19.800278 23494 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:59:19.815883 23494 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:59:19.817627 23494 catalog_manager.cc:1259] Loaded cluster ID: d5f43b24f410479c892dd4e5c00bfd87
I20260924 00:59:19.817821 23494 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260924 00:59:19.820029 23494 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f6290f559ad google::LogMessage::Fail() at ??:0
@ 0x7f6290f5888a google::LogMessage::SendToLog() at ??:0
@ 0x7f6290f554de google::LogMessage::Flush() at ??:0
@ 0x7f6290f56a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f62a6988512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f62a6985976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f62a6a0d610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f62a6a0d439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f6293ba1345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f6293ba491c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f6293ba4799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f6293b7feaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f6296e596db start_thread at ??:0
@ 0x7f628fdef61f clone at ??:0
W20260924 00:59:20.111765 23486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:59:20.112095 23486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:59:20.112259 23486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:59:20.119619 23486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:59:20.119779 23486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:59:20.119974 23486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.129
I20260924 00:59:20.129894 23486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:38521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab
--principal=kudu/127.8.64.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.129:40201
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.64.129
--webserver_port=33331
--tserver_master_addrs=127.8.64.190:45505
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.129
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:59:20.131327 23486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:59:20.132705 23486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:59:20.147073 23523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:59:20.147161 23524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:59:20.147111 23526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:59:20.148706 23486 server_base.cc:1034] running on GCE node
I20260924 00:59:20.149340 23486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:59:20.150405 23486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:59:20.151672 23486 hybrid_clock.cc:648] HybridClock initialized: now 1790211560151593 us; error 75 us; skew 500 ppm
I20260924 00:59:20.152004 23486 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:59:20 dist-test-slave-mmhs krb5kdc[22333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211560, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:59:20.155289 23486 init.cc:377] Logged in from keytab as kudu/127.8.64.129@KRBTEST.COM (short username kudu)
I20260924 00:59:20.156924 23486 webserver.cc:460] Webserver started at http://127.8.64.129:33331/ using document root <none> and password file <none>
I20260924 00:59:20.157656 23486 fs_manager.cc:359] Metadata directory not provided
I20260924 00:59:20.157904 23486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:59:20.163503 23486 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260924 00:59:20.168676 23533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:20.170145 23486 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260924 00:59:20.170377 23486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-0/wal
uuid: "471e118cd0fc492f936acf4f6258b095"
format_stamp: "Formatted at 2026-09-24 00:59:14 on dist-test-slave-mmhs"
server_key: "3eb2520249b6bc14e9ff1a51cd2887fc"
server_key_iv: "a7279037f157517a57403d81aca1653d"
server_key_version: "encryptionkey@0"
I20260924 00:59:20.171392 23486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:59:20.185065 23486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:59:20.190210 23486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:59:20.190647 23486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:59:20.192234 23486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:59:20.194358 23486 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:59:20.194482 23486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:20.194615 23486 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:59:20.194702 23486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:20.235675 23486 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.129:40201
I20260924 00:59:20.235793 23646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.129:40201 every 8 connection(s)
I20260924 00:59:20.237331 23486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
I20260924 00:59:20.238973 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 23486
I20260924 00:59:20.239629 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.130:40057
--local_ip_for_outbound_sockets=127.8.64.130
--tserver_master_addrs=127.8.64.190:45505
--webserver_port=41043
--webserver_interface=127.8.64.130
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab
--principal=kudu/127.8.64.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:38521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 00:59:20.242550 23649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:20.240379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.129:53943)
0924 00:59:20.241209 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:20.241469 (+ 260us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":337,"threads_started":1}
W20260924 00:59:20.243479 23647 heartbeater.cc:644] Failed to heartbeat to 127.8.64.190:45505 (0 consecutive failures): Network error: Failed to ping master at 127.8.64.190:45505: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
I20260924 00:59:20.244478 23649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:20.244146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.129:33897)
0924 00:59:20.244299 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:20.244369 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260924 00:59:20.245317 23649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:20.245000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.129:42673)
0924 00:59:20.245140 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:20.245220 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260924 00:59:20.245518 23647 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 00:59:20.570665 23650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:59:20.571031 23650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:59:20.571224 23650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:59:20.579037 23650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:59:20.579200 23650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:59:20.579392 23650 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.130
I20260924 00:59:20.589327 23650 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:38521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab
--principal=kudu/127.8.64.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.130:40057
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.64.130
--webserver_port=41043
--tserver_master_addrs=127.8.64.190:45505
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.130
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:59:20.590705 23650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:59:20.592051 23650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:59:20.605204 23655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:59:20.605221 23656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:59:20.605193 23658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:59:20.606122 23650 server_base.cc:1034] running on GCE node
I20260924 00:59:20.606798 23650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:59:20.607781 23650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:59:20.609035 23650 hybrid_clock.cc:648] HybridClock initialized: now 1790211560608981 us; error 74 us; skew 500 ppm
I20260924 00:59:20.609376 23650 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:59:20 dist-test-slave-mmhs krb5kdc[22333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211560, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:59:20.612599 23650 init.cc:377] Logged in from keytab as kudu/127.8.64.130@KRBTEST.COM (short username kudu)
I20260924 00:59:20.614020 23650 webserver.cc:460] Webserver started at http://127.8.64.130:41043/ using document root <none> and password file <none>
I20260924 00:59:20.614727 23650 fs_manager.cc:359] Metadata directory not provided
I20260924 00:59:20.614825 23650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:59:20.619964 23650 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260924 00:59:20.622609 23665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:20.623936 23650 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260924 00:59:20.624111 23650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-1/wal
uuid: "a9b7028f039c46259584a9163095688a"
format_stamp: "Formatted at 2026-09-24 00:59:14 on dist-test-slave-mmhs"
server_key: "d243346bce3af47d0d00dd6e2ac8b335"
server_key_iv: "86bcea2c24db8983fa633cc9081cddc7"
server_key_version: "encryptionkey@0"
I20260924 00:59:20.625085 23650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:59:20.644359 23650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:59:20.648726 23650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:59:20.648963 23650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:59:20.649904 23650 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:59:20.651285 23650 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:59:20.651372 23650 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:20.651453 23650 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:59:20.651526 23650 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:20.685961 23650 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.130:40057
I20260924 00:59:20.686002 23778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.130:40057 every 8 connection(s)
I20260924 00:59:20.687511 23650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
I20260924 00:59:20.692718 23781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:20.690886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.130:38371)
0924 00:59:20.691667 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:20.691876 (+ 209us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":304,"threads_started":1}
W20260924 00:59:20.693490 23779 heartbeater.cc:644] Failed to heartbeat to 127.8.64.190:45505 (0 consecutive failures): Network error: Failed to ping master at 127.8.64.190:45505: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
I20260924 00:59:20.694500 23781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:20.694049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.130:44167)
0924 00:59:20.694216 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:20.694354 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260924 00:59:20.695526 23781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:20.695071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.130:39837)
0924 00:59:20.695244 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:20.695342 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260924 00:59:20.695727 23779 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260924 00:59:20.695794 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 23650
I20260924 00:59:20.696405 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.131:35483
--local_ip_for_outbound_sockets=127.8.64.131
--tserver_master_addrs=127.8.64.190:45505
--webserver_port=38823
--webserver_interface=127.8.64.131
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab
--principal=kudu/127.8.64.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:38521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:59:21.017306 23782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:59:21.017638 23782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:59:21.017802 23782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:59:21.025043 23782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:59:21.025185 23782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:59:21.025370 23782 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.131
I20260924 00:59:21.035012 23782 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:38521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab
--principal=kudu/127.8.64.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.131:35483
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.64.131
--webserver_port=38823
--tserver_master_addrs=127.8.64.190:45505
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.131
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:59:21.036335 23782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:59:21.037637 23782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:59:21.050585 23787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:59:21.050650 23790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:59:21.050683 23788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:59:21.051245 23782 server_base.cc:1034] running on GCE node
I20260924 00:59:21.051857 23782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:59:21.052758 23782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:59:21.053983 23782 hybrid_clock.cc:648] HybridClock initialized: now 1790211561053927 us; error 51 us; skew 500 ppm
I20260924 00:59:21.054360 23782 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:59:21 dist-test-slave-mmhs krb5kdc[22333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211561, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:59:21.057555 23782 init.cc:377] Logged in from keytab as kudu/127.8.64.131@KRBTEST.COM (short username kudu)
I20260924 00:59:21.059033 23782 webserver.cc:460] Webserver started at http://127.8.64.131:38823/ using document root <none> and password file <none>
I20260924 00:59:21.059705 23782 fs_manager.cc:359] Metadata directory not provided
I20260924 00:59:21.059846 23782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:59:21.065126 23782 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260924 00:59:21.067975 23797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:21.069365 23782 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260924 00:59:21.069574 23782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-2/wal
uuid: "18e00ac71aed4617b1a6078ddc64d228"
format_stamp: "Formatted at 2026-09-24 00:59:15 on dist-test-slave-mmhs"
server_key: "c0a2219cbd355d4f8f49662f8f1e868c"
server_key_iv: "05b8aa2e023b621cf8fe3225a158b6f1"
server_key_version: "encryptionkey@0"
I20260924 00:59:21.070640 23782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:59:21.091773 23782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:59:21.095642 23782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:59:21.095923 23782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:59:21.096903 23782 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:59:21.098248 23782 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:59:21.098399 23782 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:21.098489 23782 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:59:21.098547 23782 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:21.132035 23782 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.131:35483
I20260924 00:59:21.132136 23909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.131:35483 every 8 connection(s)
I20260924 00:59:21.133455 23782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
I20260924 00:59:21.138685 23912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:21.136444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.131:59473)
0924 00:59:21.137228 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:21.137444 (+ 216us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":329,"threads_started":1}
I20260924 00:59:21.139307 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 23782
W20260924 00:59:21.139830 23910 heartbeater.cc:644] Failed to heartbeat to 127.8.64.190:45505 (0 consecutive failures): Network error: Failed to ping master at 127.8.64.190:45505: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
I20260924 00:59:21.141834 23912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:21.141508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.131:59523)
0924 00:59:21.141652 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:21.141719 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260924 00:59:21.142714 23912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:21.142407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.131:46829)
0924 00:59:21.142523 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:21.142598 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260924 00:59:21.142966 23910 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260924 00:59:21.246883 23914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:21.246046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.129:54635)
0924 00:59:21.246566 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:21.246645 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":303,"threads_started":1}
I20260924 00:59:21.697014 23915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:21.696228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.130:50925)
0924 00:59:21.696662 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:21.696792 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":218,"threads_started":1}
I20260924 00:59:22.144644 23916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:22.143633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.131:50779)
0924 00:59:22.144224 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:22.144345 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":271,"threads_started":1}
I20260924 00:59:22.248416 23917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:22.247715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.129:57875)
0924 00:59:22.248177 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:22.248256 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":242,"threads_started":1}
I20260924 00:59:22.698637 23918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:22.697847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.130:56191)
0924 00:59:22.698376 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:22.698461 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":236,"threads_started":1}
I20260924 00:59:23.146806 23919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:23.145910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.131:51377)
0924 00:59:23.146496 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:23.146592 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":296,"threads_started":1}
I20260924 00:59:23.249982 23920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:23.249295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.129:59629)
0924 00:59:23.249732 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:23.249813 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":239,"threads_started":1}
I20260924 00:59:23.700270 23921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:23.699568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.130:42037)
0924 00:59:23.699999 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:23.700079 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":227,"threads_started":1}
I20260924 00:59:24.148370 23922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:24.147689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.131:48709)
0924 00:59:24.148120 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:24.148197 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":221,"threads_started":1}
I20260924 00:59:24.251724 23923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:24.250969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.129:49217)
0924 00:59:24.251436 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:24.251529 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":244,"threads_started":1}
I20260924 00:59:24.701931 23924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:24.701172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.130:33501)
0924 00:59:24.701641 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:24.701736 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":240,"threads_started":1}
I20260924 00:59:25.149899 23925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:25.149203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.131:52041)
0924 00:59:25.149617 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:25.149705 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":204,"threads_started":1}
I20260924 00:59:25.253561 23926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:25.252734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.129:53875)
0924 00:59:25.253272 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:25.253369 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":295,"threads_started":1}
I20260924 00:59:25.703646 23927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:25.702877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.130:37375)
0924 00:59:25.703344 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:25.703428 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":233,"threads_started":1}
I20260924 00:59:26.151544 23928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:26.150849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.131:54471)
0924 00:59:26.151295 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:26.151375 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":224,"threads_started":1}
I20260924 00:59:26.255597 23929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:26.254734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.129:48979)
0924 00:59:26.255335 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:26.255431 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":380,"threads_started":1}
I20260924 00:59:26.705360 23930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:26.704568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.130:59587)
0924 00:59:26.705090 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:26.705192 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":235,"threads_started":1}
I20260924 00:59:27.153168 23931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:27.152462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.131:54595)
0924 00:59:27.152912 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:27.152999 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":243,"threads_started":1}
I20260924 00:59:27.257361 23932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:27.256522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.129:51903)
0924 00:59:27.257076 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:27.257158 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":253,"threads_started":1}
I20260924 00:59:27.707459 23933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:27.706513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.130:46647)
0924 00:59:27.707046 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:27.707157 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":251,"threads_started":1}
I20260924 00:59:28.154928 23934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:28.154116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.131:54527)
0924 00:59:28.154639 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:28.154717 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":261,"threads_started":1}
I20260924 00:59:28.259218 23935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:28.258391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.129:42111)
0924 00:59:28.258885 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:28.258989 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":225,"threads_started":1}
I20260924 00:59:28.709043 23936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:28.708342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.130:40383)
0924 00:59:28.708795 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:28.708875 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":245,"threads_started":1}
I20260924 00:59:29.156411 23937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:29.155740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.131:54173)
0924 00:59:29.156159 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:29.156247 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":223,"threads_started":1}
I20260924 00:59:29.260936 23938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:29.260206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.129:53729)
0924 00:59:29.260677 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:29.260759 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":242,"threads_started":1}
I20260924 00:59:29.710922 23939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:29.710115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.130:48775)
0924 00:59:29.710657 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:29.710740 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":294,"threads_started":1}
I20260924 00:59:30.158084 23940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:30.157340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.131:35129)
0924 00:59:30.157830 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:30.157912 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":247,"threads_started":1}
I20260924 00:59:30.262642 23941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:30.261843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.129:58111)
0924 00:59:30.262375 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:30.262457 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":264,"threads_started":1}
I20260924 00:59:30.712481 23942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:30.711814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.130:39705)
0924 00:59:30.712242 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:30.712321 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":225,"threads_started":1}
I20260924 00:59:31.159745 23943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:31.159012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.131:56413)
0924 00:59:31.159439 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:31.159520 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":238,"threads_started":1}
I20260924 00:59:31.264230 23944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:31.263515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.129:35143)
0924 00:59:31.263961 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:31.264041 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":225,"threads_started":1}
I20260924 00:59:31.714196 23945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:31.713434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.130:47555)
0924 00:59:31.713918 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:31.714006 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":264,"threads_started":1}
I20260924 00:59:32.161408 23946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:32.160664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.131:52303)
0924 00:59:32.161136 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:32.161230 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":248,"threads_started":1}
I20260924 00:59:32.265944 23947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:32.265211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.129:57053)
0924 00:59:32.265702 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:32.265787 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":246,"threads_started":1}
I20260924 00:59:32.715878 23948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:32.715176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.130:56237)
0924 00:59:32.715632 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:32.715722 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":246,"threads_started":1}
I20260924 00:59:33.163054 23949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:33.162396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.131:59949)
0924 00:59:33.162819 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:33.162898 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":213,"threads_started":1}
I20260924 00:59:33.267613 23950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:33.266868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.129:60363)
0924 00:59:33.267321 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:33.267432 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":239,"threads_started":1}
I20260924 00:59:33.717536 23951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:33.716742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.130:55365)
0924 00:59:33.717254 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:33.717348 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":259,"threads_started":1}
I20260924 00:59:34.164726 23952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:34.163963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.131:60305)
0924 00:59:34.164418 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:34.164510 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":223,"threads_started":1}
I20260924 00:59:34.269282 23953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:34.268593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.129:40015)
0924 00:59:34.269025 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:34.269106 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":215,"threads_started":1}
I20260924 00:59:34.719144 23954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:34.718439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.130:58735)
0924 00:59:34.718874 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:34.718962 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":229,"threads_started":1}
I20260924 00:59:35.166414 23955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:35.165567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.131:43131)
0924 00:59:35.166069 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:35.166165 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":299,"threads_started":1}
I20260924 00:59:35.271390 23956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:35.270552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.129:53905)
0924 00:59:35.271046 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:35.271182 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":239,"threads_started":1}
I20260924 00:59:35.720973 23957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:35.720231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.130:54529)
0924 00:59:35.720691 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:35.720789 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":222,"threads_started":1}
I20260924 00:59:36.151528 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 23486
I20260924 00:59:36.168385 23959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:36.167496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45505 (local address 127.8.64.131:40333)
0924 00:59:36.168036 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:36.168214 (+ 178us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":297,"threads_started":1}
I20260924 00:59:36.170130 23958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:36.152636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47298 (local address 127.8.64.129:40201)
0924 00:59:36.153714 (+ 1078us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:36.153729 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:36.153788 (+ 59us) server_negotiation.cc:407] Connection header received
0924 00:59:36.155655 (+ 1867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:36.155664 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:36.156080 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:36.156389 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:36.157746 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:36.159087 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:36.160287 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:36.160931 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:36.161069 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:36.163862 (+ 2793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:36.163870 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:36.163884 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:36.163918 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:36.166701 (+ 2783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:36.167429 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:36.167435 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:36.167441 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:36.167607 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:36.168046 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:36.168053 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:36.168056 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:36.168672 (+ 616us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:36.168985 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:36.169439 (+ 454us) server_negotiation.cc:299] Negotiation successful
0924 00:59:36.169864 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":351,"threads_started":1}
I20260924 00:59:36.171550 23642 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:59:36.255453 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 23486
I20260924 00:59:36.272893 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 23650
I20260924 00:59:36.290625 23961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:36.273837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52958 (local address 127.8.64.130:40057)
0924 00:59:36.274428 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:36.274442 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:36.274479 (+ 37us) server_negotiation.cc:407] Connection header received
0924 00:59:36.276087 (+ 1608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:36.276097 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:36.276561 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:36.276919 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:36.278515 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:36.279735 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:36.280987 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:36.281643 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:36.281858 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:36.284421 (+ 2563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:36.284429 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:36.284444 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:36.284480 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:36.287267 (+ 2787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:36.287954 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:36.287961 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:36.287968 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:36.288129 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:36.288521 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:36.288527 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:36.288530 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:36.289037 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:36.289364 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:36.289797 (+ 433us) server_negotiation.cc:299] Negotiation successful
0924 00:59:36.290212 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":250,"threads_started":1}
I20260924 00:59:36.292023 23774 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:59:36.375159 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 23650
I20260924 00:59:36.393313 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 23782
I20260924 00:59:36.411300 23963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:36.394432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44380 (local address 127.8.64.131:35483)
0924 00:59:36.395030 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:36.395044 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:36.395084 (+ 40us) server_negotiation.cc:407] Connection header received
0924 00:59:36.396688 (+ 1604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:36.396698 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:36.397140 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:36.397474 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:36.398884 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:36.400096 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:36.401552 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:36.402203 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:36.402401 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:36.405192 (+ 2791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:36.405199 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:36.405215 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:36.405251 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:36.408048 (+ 2797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:36.408791 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:36.408801 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:36.408810 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:36.408984 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:36.409330 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:36.409334 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:36.409336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:36.409858 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:36.410092 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:36.410654 (+ 562us) server_negotiation.cc:299] Negotiation successful
0924 00:59:36.410996 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":287,"threads_started":1}
I20260924 00:59:36.412640 23905 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:59:36.495642 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 23782
2026-09-24T00:59:36Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24336 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:59:36 dist-test-slave-mmhs krb5kdc[23967](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:59:36 dist-test-slave-mmhs krb5kdc[23967](info): set up 2 sockets
Sep 24 00:59:36 dist-test-slave-mmhs krb5kdc[23967](info): commencing operation
krb5kdc: starting...
W20260924 00:59:37.620532 8450 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Sep 24 00:59:37 dist-test-slave-mmhs krb5kdc[23967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211577, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:59:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:59:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.64.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
I20260924 00:59:37.870966 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:42983
--webserver_interface=127.8.64.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:35317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:42983
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:59:38.196780 23980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:59:38.197114 23980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:59:38.197340 23980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:59:38.205502 23980 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260924 00:59:38.205588 23980 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260924 00:59:38.205665 23980 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260924 00:59:38.205739 23980 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:59:38.205808 23980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:59:38.205930 23980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:59:38.206008 23980 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:59:38.206086 23980 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:59:38.215862 23980 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:35317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:42983
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:42983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:59:38.217108 23980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:59:38.218451 23980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:59:38.230456 23985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:59:38.230398 23986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:59:38.230500 23988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:59:38.230803 23980 server_base.cc:1034] running on GCE node
I20260924 00:59:38.231902 23980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:59:38.233354 23980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:59:38.234586 23980 hybrid_clock.cc:648] HybridClock initialized: now 1790211578234505 us; error 57 us; skew 500 ppm
I20260924 00:59:38.234937 23980 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:59:38 dist-test-slave-mmhs krb5kdc[23967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211578, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:59:38.237973 23980 init.cc:377] Logged in from keytab as kudu/127.8.64.190@KRBTEST.COM (short username kudu)
I20260924 00:59:38.239499 23980 webserver.cc:460] Webserver started at http://127.8.64.190:34309/ using document root <none> and password file <none>
I20260924 00:59:38.240180 23980 fs_manager.cc:359] Metadata directory not provided
I20260924 00:59:38.240312 23980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:59:38.240630 23980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:59:38.243594 23980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "1ffc1d56a39440f09aa42a5e0174c770"
format_stamp: "Formatted at 2026-09-24 00:59:38 on dist-test-slave-mmhs"
server_key: "04e8f38f2e0aaffd2381c2ef1e98b816"
server_key_iv: "6ecfc60af2ee8cc0642ed76bbe6f88ce"
server_key_version: "encryptionkey@0"
I20260924 00:59:38.244410 23980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "1ffc1d56a39440f09aa42a5e0174c770"
format_stamp: "Formatted at 2026-09-24 00:59:38 on dist-test-slave-mmhs"
server_key: "04e8f38f2e0aaffd2381c2ef1e98b816"
server_key_iv: "6ecfc60af2ee8cc0642ed76bbe6f88ce"
server_key_version: "encryptionkey@0"
I20260924 00:59:38.249389 23980 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260924 00:59:38.252909 23995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:38.254266 23980 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260924 00:59:38.254503 23980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "1ffc1d56a39440f09aa42a5e0174c770"
format_stamp: "Formatted at 2026-09-24 00:59:38 on dist-test-slave-mmhs"
server_key: "04e8f38f2e0aaffd2381c2ef1e98b816"
server_key_iv: "6ecfc60af2ee8cc0642ed76bbe6f88ce"
server_key_version: "encryptionkey@0"
I20260924 00:59:38.254719 23980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:59:38.297097 23980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:59:38.301097 23980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:59:38.301400 23980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:59:38.319949 23980 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.190:42983
I20260924 00:59:38.319962 24046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.190:42983 every 8 connection(s)
I20260924 00:59:38.321336 23980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 00:59:38.324658 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 23980
I20260924 00:59:38.324878 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 00:59:38.325466 24047 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:59:38.325748 8450 external_mini_cluster.cc:1428] Setting key 2ec2d9a5042085d709abe8c534b2923c
I20260924 00:59:38.340229 24047 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770: Bootstrap starting.
Sep 24 00:59:38 dist-test-slave-mmhs krb5kdc[23967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211577, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:59:38.343326 24047 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770: Neither blocks nor log segments found. Creating new log.
I20260924 00:59:38.344377 24047 log.cc:824] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770: Log is configured to *not* fsync() on all Append() calls
I20260924 00:59:38.347049 24047 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770: No bootstrap required, opened a new log
I20260924 00:59:38.348750 24050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:38.327790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60352 (local address 127.8.64.190:42983)
0924 00:59:38.328598 (+ 808us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:38.328612 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:38.328651 (+ 39us) server_negotiation.cc:407] Connection header received
0924 00:59:38.330370 (+ 1719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:38.330392 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:38.330766 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:38.331082 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:38.332796 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:38.334044 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:38.335980 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:38.336758 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:38.336888 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:38.341685 (+ 4797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:38.341694 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:38.341707 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:38.341748 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:38.344851 (+ 3103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:38.345546 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:38.345553 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:38.345558 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:38.345721 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:38.346102 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:38.346108 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:38.346111 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:38.346652 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:38.346949 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:38.347360 (+ 411us) server_negotiation.cc:299] Negotiation successful
0924 00:59:38.347783 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":383,"threads_started":1}
I20260924 00:59:38.355020 24047 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ffc1d56a39440f09aa42a5e0174c770" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42983 } }
I20260924 00:59:38.355517 24047 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:59:38.355634 24047 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ffc1d56a39440f09aa42a5e0174c770, State: Initialized, Role: FOLLOWER
I20260924 00:59:38.356288 24047 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ffc1d56a39440f09aa42a5e0174c770" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42983 } }
I20260924 00:59:38.356503 24047 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:59:38.356603 24047 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:59:38.356722 24047 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:59:38.359216 24047 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ffc1d56a39440f09aa42a5e0174c770" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42983 } }
I20260924 00:59:38.359699 24047 leader_election.cc:302] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ffc1d56a39440f09aa42a5e0174c770; no voters:
I20260924 00:59:38.360231 24047 leader_election.cc:288] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:59:38.360356 24052 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:59:38.361079 24052 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [term 1 LEADER]: Becoming Leader. State: Replica: 1ffc1d56a39440f09aa42a5e0174c770, State: Running, Role: LEADER
I20260924 00:59:38.361641 24052 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ffc1d56a39440f09aa42a5e0174c770" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42983 } }
I20260924 00:59:38.362248 24047 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:59:38.364423 24054 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ffc1d56a39440f09aa42a5e0174c770. Latest consensus state: current_term: 1 leader_uuid: "1ffc1d56a39440f09aa42a5e0174c770" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ffc1d56a39440f09aa42a5e0174c770" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42983 } } }
I20260924 00:59:38.364398 24053 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ffc1d56a39440f09aa42a5e0174c770" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ffc1d56a39440f09aa42a5e0174c770" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42983 } } }
I20260924 00:59:38.364864 24054 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [sys.catalog]: This master's current role is: LEADER
I20260924 00:59:38.364951 24053 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [sys.catalog]: This master's current role is: LEADER
I20260924 00:59:38.368345 24057 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:59:38.374550 24057 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:59:38.385355 24057 catalog_manager.cc:1347] Generated new cluster ID: 6318508f6afc4040b7443bf314edebcd
I20260924 00:59:38.385476 24057 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:59:38.396540 24057 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:59:38.397852 24057 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:59:38.408463 24057 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770: Generated new TSK 0
I20260924 00:59:38.409312 24057 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.64.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab.
Entry for principal kudu/127.8.64.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab.
I20260924 00:59:38.470386 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.129:0
--local_ip_for_outbound_sockets=127.8.64.129
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:42983
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab
--principal=kudu/127.8.64.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:35317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:59:38.802359 24073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:59:38.802702 24073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:59:38.802901 24073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:59:38.810261 24073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:59:38.810416 24073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:59:38.810606 24073 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.129
I20260924 00:59:38.820474 24073 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:35317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab
--principal=kudu/127.8.64.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:42983
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.129
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:59:38.821756 24073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:59:38.823210 24073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:59:38.836297 24078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:59:38.836325 24079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:59:38.836357 24081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:59:38.836984 24073 server_base.cc:1034] running on GCE node
I20260924 00:59:38.837616 24073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:59:38.838652 24073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:59:38.839881 24073 hybrid_clock.cc:648] HybridClock initialized: now 1790211578839843 us; error 50 us; skew 500 ppm
I20260924 00:59:38.840191 24073 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:59:38 dist-test-slave-mmhs krb5kdc[23967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211578, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:59:38.843175 24073 init.cc:377] Logged in from keytab as kudu/127.8.64.129@KRBTEST.COM (short username kudu)
I20260924 00:59:38.844605 24073 webserver.cc:460] Webserver started at http://127.8.64.129:36865/ using document root <none> and password file <none>
I20260924 00:59:38.845253 24073 fs_manager.cc:359] Metadata directory not provided
I20260924 00:59:38.845402 24073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:59:38.845717 24073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:59:38.848572 24073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-0/data/instance:
uuid: "4727a59d1809474dad8a93ba1bf545be"
format_stamp: "Formatted at 2026-09-24 00:59:38 on dist-test-slave-mmhs"
server_key: "95872cadad7a1c8ac90e941ca2919790"
server_key_iv: "40db947aad7907bb11b9f1d79ce224b1"
server_key_version: "encryptionkey@0"
I20260924 00:59:38.849310 24073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance:
uuid: "4727a59d1809474dad8a93ba1bf545be"
format_stamp: "Formatted at 2026-09-24 00:59:38 on dist-test-slave-mmhs"
server_key: "95872cadad7a1c8ac90e941ca2919790"
server_key_iv: "40db947aad7907bb11b9f1d79ce224b1"
server_key_version: "encryptionkey@0"
I20260924 00:59:38.854393 24073 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.007s
I20260924 00:59:38.857797 24088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:38.859241 24073 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 00:59:38.859414 24073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-0/wal
uuid: "4727a59d1809474dad8a93ba1bf545be"
format_stamp: "Formatted at 2026-09-24 00:59:38 on dist-test-slave-mmhs"
server_key: "95872cadad7a1c8ac90e941ca2919790"
server_key_iv: "40db947aad7907bb11b9f1d79ce224b1"
server_key_version: "encryptionkey@0"
I20260924 00:59:38.859665 24073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:59:38.895495 24073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:59:38.899684 24073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:59:38.899993 24073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:59:38.900970 24073 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:59:38.902377 24073 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:59:38.902482 24073 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:38.902650 24073 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:59:38.902735 24073 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:38.935982 24073 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.129:37443
I20260924 00:59:38.936035 24200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.129:37443 every 8 connection(s)
I20260924 00:59:38.937288 24073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
I20260924 00:59:38.945024 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 24073
I20260924 00:59:38.945289 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance
I20260924 00:59:38.945940 8450 external_mini_cluster.cc:1428] Setting key bfad0687875036a0e324be3688bbbdba
Sep 24 00:59:38 dist-test-slave-mmhs krb5kdc[23967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211578, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:59:38.959147 24203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:38.939997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:45023 (local address 127.8.64.190:42983)
0924 00:59:38.940428 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:38.940433 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:38.941358 (+ 925us) server_negotiation.cc:407] Connection header received
0924 00:59:38.943242 (+ 1884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:38.943248 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:38.943340 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:38.943464 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:38.945081 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:38.945936 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:38.947963 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:38.948359 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:38.948458 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:38.952850 (+ 4392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:38.952855 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:38.952858 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:38.952891 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:38.955483 (+ 2592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:38.956428 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:38.956432 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:38.956434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:38.956550 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:38.956989 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:38.956996 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:38.957001 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:38.957386 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:38.957546 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:38.958316 (+ 770us) server_negotiation.cc:299] Negotiation successful
0924 00:59:38.958618 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":218,"threads_started":1}
I20260924 00:59:38.959864 24204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:38.940534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.129:45023)
0924 00:59:38.941201 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:38.941244 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 00:59:38.942977 (+ 1733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:59:38.943703 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:59:38.943711 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:59:38.944192 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:59:38.944883 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:38.944910 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:38.946140 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:38.946145 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:38.947793 (+ 1648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:38.947805 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:38.948599 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:38.948607 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:38.948937 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:38.949649 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:59:38.949689 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:59:38.952629 (+ 2940us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:59:38.955635 (+ 3006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:59:38.955646 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:59:38.955666 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:59:38.956303 (+ 637us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:59:38.956677 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:59:38.956682 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:59:38.956686 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:59:38.956865 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:59:38.957512 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:59:38.957518 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:59:38.957901 (+ 383us) client_negotiation.cc:769] Sending connection context
0924 00:59:38.958124 (+ 223us) client_negotiation.cc:240] Negotiation successful
0924 00:59:38.958650 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":277,"threads_started":1}
I20260924 00:59:38.961423 24201 heartbeater.cc:347] Connected to a master server at 127.8.64.190:42983
I20260924 00:59:38.961733 24201 heartbeater.cc:464] Registering TS with master...
I20260924 00:59:38.962519 24201 heartbeater.cc:511] Master 127.8.64.190:42983 requested a full tablet report, sending...
I20260924 00:59:38.964730 24012 ts_manager.cc:194] Registered new tserver with Master: 4727a59d1809474dad8a93ba1bf545be (127.8.64.129:37443)
I20260924 00:59:38.966181 24012 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.129@KRBTEST.COM'} at 127.8.64.129:45023
WARNING: no policy specified for kudu/127.8.64.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab.
Entry for principal kudu/127.8.64.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab.
I20260924 00:59:39.003985 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.130:0
--local_ip_for_outbound_sockets=127.8.64.130
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:42983
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab
--principal=kudu/127.8.64.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:35317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:59:39.332983 24207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:59:39.333313 24207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:59:39.333493 24207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:59:39.340873 24207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:59:39.341030 24207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:59:39.341228 24207 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.130
I20260924 00:59:39.351006 24207 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:35317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab
--principal=kudu/127.8.64.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:42983
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.130
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:59:39.352349 24207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:59:39.353717 24207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:59:39.367409 24212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:59:39.367370 24213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:59:39.367470 24215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:59:39.368543 24207 server_base.cc:1034] running on GCE node
I20260924 00:59:39.369153 24207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:59:39.370076 24207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:59:39.371321 24207 hybrid_clock.cc:648] HybridClock initialized: now 1790211579371255 us; error 85 us; skew 500 ppm
I20260924 00:59:39.371634 24207 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:59:39 dist-test-slave-mmhs krb5kdc[23967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211579, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:59:39.374802 24207 init.cc:377] Logged in from keytab as kudu/127.8.64.130@KRBTEST.COM (short username kudu)
I20260924 00:59:39.376346 24207 webserver.cc:460] Webserver started at http://127.8.64.130:33007/ using document root <none> and password file <none>
I20260924 00:59:39.377051 24207 fs_manager.cc:359] Metadata directory not provided
I20260924 00:59:39.377158 24207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:59:39.377417 24207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:59:39.380306 24207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-1/data/instance:
uuid: "ae77c9ce9b674ebbb2f57f0bcc293036"
format_stamp: "Formatted at 2026-09-24 00:59:39 on dist-test-slave-mmhs"
server_key: "7ee3772027126ed8f50e0a5594458d7b"
server_key_iv: "867be1ef8dde9ee0daf38d095f63ce6c"
server_key_version: "encryptionkey@0"
I20260924 00:59:39.381001 24207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance:
uuid: "ae77c9ce9b674ebbb2f57f0bcc293036"
format_stamp: "Formatted at 2026-09-24 00:59:39 on dist-test-slave-mmhs"
server_key: "7ee3772027126ed8f50e0a5594458d7b"
server_key_iv: "867be1ef8dde9ee0daf38d095f63ce6c"
server_key_version: "encryptionkey@0"
I20260924 00:59:39.385986 24207 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260924 00:59:39.389436 24222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:39.390833 24207 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 00:59:39.390997 24207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-1/wal
uuid: "ae77c9ce9b674ebbb2f57f0bcc293036"
format_stamp: "Formatted at 2026-09-24 00:59:39 on dist-test-slave-mmhs"
server_key: "7ee3772027126ed8f50e0a5594458d7b"
server_key_iv: "867be1ef8dde9ee0daf38d095f63ce6c"
server_key_version: "encryptionkey@0"
I20260924 00:59:39.391180 24207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:59:39.416913 24207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:59:39.421175 24207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:59:39.421468 24207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:59:39.422513 24207 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:59:39.423808 24207 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:59:39.423879 24207 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:39.424054 24207 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:59:39.424105 24207 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:39.457387 24207 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.130:43121
I20260924 00:59:39.457516 24334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.130:43121 every 8 connection(s)
I20260924 00:59:39.458757 24207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
I20260924 00:59:39.468724 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 24207
I20260924 00:59:39.468940 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance
I20260924 00:59:39.469636 8450 external_mini_cluster.cc:1428] Setting key 54c95d0a0d3844f2df24207fbe6fa751
Sep 24 00:59:39 dist-test-slave-mmhs krb5kdc[23967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211579, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.130@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:59:39.479638 24337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:39.461530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:45739 (local address 127.8.64.190:42983)
0924 00:59:39.462011 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:39.462016 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:39.463044 (+ 1028us) server_negotiation.cc:407] Connection header received
0924 00:59:39.465012 (+ 1968us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:39.465018 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:39.465113 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:39.465245 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:39.466797 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:39.467542 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:39.469167 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:39.469844 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:39.469996 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:39.473627 (+ 3631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:39.473632 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:39.473635 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:39.473667 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:39.476533 (+ 2866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:39.477271 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:39.477281 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:39.477284 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:39.477407 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:39.477860 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:39.477870 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:39.477872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:39.478127 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:39.478324 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:39.479015 (+ 691us) server_negotiation.cc:299] Negotiation successful
0924 00:59:39.479265 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":215,"threads_started":1}
I20260924 00:59:39.480549 24338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:39.462075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.130:45739)
0924 00:59:39.462807 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:39.462895 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0924 00:59:39.464716 (+ 1821us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:59:39.465478 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:59:39.465486 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:59:39.465985 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:59:39.466624 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:39.466640 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:39.467708 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:39.467714 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:39.468989 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:39.469001 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:39.470074 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:39.470079 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:39.470284 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:39.471045 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:59:39.471074 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:59:39.473399 (+ 2325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:59:39.476688 (+ 3289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:59:39.476696 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:59:39.476713 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:59:39.477131 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:59:39.477521 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:59:39.477527 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:59:39.477531 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:59:39.477732 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:59:39.478266 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:59:39.478275 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:59:39.478641 (+ 366us) client_negotiation.cc:769] Sending connection context
0924 00:59:39.478919 (+ 278us) client_negotiation.cc:240] Negotiation successful
0924 00:59:39.479441 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":316,"threads_started":1}
I20260924 00:59:39.481978 24335 heartbeater.cc:347] Connected to a master server at 127.8.64.190:42983
I20260924 00:59:39.482316 24335 heartbeater.cc:464] Registering TS with master...
I20260924 00:59:39.483163 24335 heartbeater.cc:511] Master 127.8.64.190:42983 requested a full tablet report, sending...
I20260924 00:59:39.484752 24012 ts_manager.cc:194] Registered new tserver with Master: ae77c9ce9b674ebbb2f57f0bcc293036 (127.8.64.130:43121)
I20260924 00:59:39.485581 24012 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.130@KRBTEST.COM'} at 127.8.64.130:45739
WARNING: no policy specified for kudu/127.8.64.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab.
Entry for principal kudu/127.8.64.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab.
I20260924 00:59:39.526695 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.131:0
--local_ip_for_outbound_sockets=127.8.64.131
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:42983
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab
--principal=kudu/127.8.64.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:35317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:59:39.851853 24341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:59:39.852134 24341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:59:39.852305 24341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:59:39.860095 24341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:59:39.860201 24341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:59:39.860366 24341 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.131
I20260924 00:59:39.870260 24341 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:35317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab
--principal=kudu/127.8.64.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:42983
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.131
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 00:59:39.871613 24341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:59:39.872990 24341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:59:39.886662 24349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:59:39.886737 24346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:59:39.886618 24347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:59:39.887437 24341 server_base.cc:1034] running on GCE node
I20260924 00:59:39.888010 24341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:59:39.888947 24341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:59:39.890162 24341 hybrid_clock.cc:648] HybridClock initialized: now 1790211579890115 us; error 39 us; skew 500 ppm
I20260924 00:59:39.890527 24341 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 00:59:39 dist-test-slave-mmhs krb5kdc[23967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211579, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:59:39.893847 24341 init.cc:377] Logged in from keytab as kudu/127.8.64.131@KRBTEST.COM (short username kudu)
I20260924 00:59:39.895712 24341 webserver.cc:460] Webserver started at http://127.8.64.131:43801/ using document root <none> and password file <none>
I20260924 00:59:39.896505 24341 fs_manager.cc:359] Metadata directory not provided
I20260924 00:59:39.896620 24341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:59:39.896908 24341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:59:39.900163 24341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-2/data/instance:
uuid: "4a1609a2bbb742e4b6722861bd63c4a8"
format_stamp: "Formatted at 2026-09-24 00:59:39 on dist-test-slave-mmhs"
server_key: "1af4b25f4ad11c0ca38b5f35e4758e9a"
server_key_iv: "4e2c781d84beede4e8e20109638f010f"
server_key_version: "encryptionkey@0"
I20260924 00:59:39.900933 24341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance:
uuid: "4a1609a2bbb742e4b6722861bd63c4a8"
format_stamp: "Formatted at 2026-09-24 00:59:39 on dist-test-slave-mmhs"
server_key: "1af4b25f4ad11c0ca38b5f35e4758e9a"
server_key_iv: "4e2c781d84beede4e8e20109638f010f"
server_key_version: "encryptionkey@0"
I20260924 00:59:39.906704 24341 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260924 00:59:39.910579 24356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:39.912019 24341 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260924 00:59:39.912199 24341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-2/wal
uuid: "4a1609a2bbb742e4b6722861bd63c4a8"
format_stamp: "Formatted at 2026-09-24 00:59:39 on dist-test-slave-mmhs"
server_key: "1af4b25f4ad11c0ca38b5f35e4758e9a"
server_key_iv: "4e2c781d84beede4e8e20109638f010f"
server_key_version: "encryptionkey@0"
I20260924 00:59:39.912389 24341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:59:39.933585 24341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:59:39.938208 24341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:59:39.938551 24341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:59:39.939589 24341 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:59:39.941052 24341 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:59:39.941161 24341 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:39.941336 24341 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:59:39.941409 24341 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:59:39.969933 24201 heartbeater.cc:503] Master 127.8.64.190:42983 was elected leader, sending a full tablet report...
I20260924 00:59:39.975418 24341 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.131:38205
I20260924 00:59:39.975459 24468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.131:38205 every 8 connection(s)
I20260924 00:59:39.976878 24341 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
I20260924 00:59:39.980885 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 24341
I20260924 00:59:39.981101 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance
I20260924 00:59:39.981683 8450 external_mini_cluster.cc:1428] Setting key 30de987560fb362689a1751fce5fa4b0
Sep 24 00:59:39 dist-test-slave-mmhs krb5kdc[23967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211579, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.131@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 00:59:39.996917 24337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:39.979514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:37701 (local address 127.8.64.190:42983)
0924 00:59:39.979720 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:39.979725 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:39.980897 (+ 1172us) server_negotiation.cc:407] Connection header received
0924 00:59:39.983142 (+ 2245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:39.983148 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:39.983239 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:39.983351 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:39.985061 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:39.985803 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:39.987349 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:39.987720 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:39.987812 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:39.991212 (+ 3400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:39.991217 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:39.991220 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:39.991254 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:39.993596 (+ 2342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:39.994252 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:39.994255 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:39.994258 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:39.994461 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:39.995027 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:39.995032 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:39.995034 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:39.995372 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:39.995580 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:39.996324 (+ 744us) server_negotiation.cc:299] Negotiation successful
0924 00:59:39.996547 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260924 00:59:39.997926 24471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:39.980026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.131:37701)
0924 00:59:39.980743 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:39.980785 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 00:59:39.982824 (+ 2039us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:59:39.983592 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:59:39.983603 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:59:39.984187 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:59:39.984899 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:39.984919 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:39.985929 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:39.985933 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:39.987227 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:39.987239 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:39.987907 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:39.987912 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:39.988099 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:39.988806 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:59:39.988832 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:59:39.991016 (+ 2184us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:59:39.993723 (+ 2707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:59:39.993730 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:59:39.993742 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:59:39.994145 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:59:39.994555 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:59:39.994558 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:59:39.994561 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:59:39.994760 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:59:39.995538 (+ 778us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:59:39.995548 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:59:39.995897 (+ 349us) client_negotiation.cc:769] Sending connection context
0924 00:59:39.996221 (+ 324us) client_negotiation.cc:240] Negotiation successful
0924 00:59:39.996717 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":285,"threads_started":1}
I20260924 00:59:39.999189 24469 heartbeater.cc:347] Connected to a master server at 127.8.64.190:42983
I20260924 00:59:39.999455 24469 heartbeater.cc:464] Registering TS with master...
I20260924 00:59:40.000089 24469 heartbeater.cc:511] Master 127.8.64.190:42983 requested a full tablet report, sending...
I20260924 00:59:40.001384 24012 ts_manager.cc:194] Registered new tserver with Master: 4a1609a2bbb742e4b6722861bd63c4a8 (127.8.64.131:38205)
I20260924 00:59:40.002097 24012 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.131@KRBTEST.COM'} at 127.8.64.131:37701
I20260924 00:59:40.009750 8450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 00:59:40.027812 24337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:40.014169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60360 (local address 127.8.64.190:42983)
0924 00:59:40.014409 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:40.014417 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:40.014846 (+ 429us) server_negotiation.cc:407] Connection header received
0924 00:59:40.015065 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:40.015080 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:40.015141 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:40.015247 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:40.016753 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:40.017573 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:40.019046 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:40.019669 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:40.019804 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:40.022260 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:40.022267 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:40.022271 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:40.022367 (+ 96us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:40.024927 (+ 2560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:40.025674 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:40.025681 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:40.025685 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:40.025829 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:40.026225 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:40.026232 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:40.026238 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:40.026603 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:40.026877 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:40.027212 (+ 335us) server_negotiation.cc:299] Negotiation successful
0924 00:59:40.027437 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260924 00:59:40.031862 24012 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60360:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:59:40.038771 24012 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:59:40.074182 24481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:40.050674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:37443 (local address 127.0.0.1:43106)
0924 00:59:40.064831 (+ 14157us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:40.064911 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0924 00:59:40.065145 (+ 234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:59:40.066175 (+ 1030us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:59:40.066183 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:59:40.066223 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:59:40.066895 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:40.066920 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:40.069035 (+ 2115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:40.069040 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:40.071397 (+ 2357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:40.071409 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:40.073058 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:40.073067 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:40.073262 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:40.073316 (+ 54us) client_negotiation.cc:769] Sending connection context
0924 00:59:40.073447 (+ 131us) client_negotiation.cc:240] Negotiation successful
0924 00:59:40.073668 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13883,"mutex_wait_us":130,"spinlock_wait_cycles":6528,"thread_start_us":13682,"threads_started":1}
I20260924 00:59:40.074703 24482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:40.050765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43106 (local address 127.8.64.129:37443)
0924 00:59:40.064007 (+ 13242us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:40.064020 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:40.065210 (+ 1190us) server_negotiation.cc:407] Connection header received
0924 00:59:40.065405 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:40.065413 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:40.065674 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:40.065867 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:59:40.067155 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:40.068840 (+ 1685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:40.071552 (+ 2712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:40.072840 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:40.072985 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:40.073244 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:40.073549 (+ 305us) server_negotiation.cc:299] Negotiation successful
0924 00:59:40.074344 (+ 795us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12992,"thread_start_us":13256,"threads_started":1}
I20260924 00:59:40.075032 24483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:40.050674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.131:38205 (local address 127.0.0.1:55196)
0924 00:59:40.064831 (+ 14157us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:40.064913 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0924 00:59:40.065145 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:59:40.066175 (+ 1030us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:59:40.066201 (+ 26us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:59:40.066233 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:59:40.066930 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:40.066942 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:40.068993 (+ 2051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:40.068999 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:40.071995 (+ 2996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:40.072010 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:40.074075 (+ 2065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:40.074084 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:40.074349 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:40.074381 (+ 32us) client_negotiation.cc:769] Sending connection context
0924 00:59:40.074455 (+ 74us) client_negotiation.cc:240] Negotiation successful
0924 00:59:40.074602 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13924,"mutex_wait_us":329,"spinlock_wait_cycles":33024,"thread_start_us":13552,"threads_started":1}
I20260924 00:59:40.075588 24484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:40.051208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55196 (local address 127.8.64.131:38205)
0924 00:59:40.063891 (+ 12683us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:40.063906 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:40.065293 (+ 1387us) server_negotiation.cc:407] Connection header received
0924 00:59:40.065404 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:40.065413 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:40.065675 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:40.065867 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:59:40.067186 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:40.068849 (+ 1663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:40.072174 (+ 3325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:40.073594 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:40.073712 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:40.073923 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:40.075074 (+ 1151us) server_negotiation.cc:299] Negotiation successful
0924 00:59:40.075255 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12382,"thread_start_us":12705,"threads_started":1}
I20260924 00:59:40.075870 24486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:40.066260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.130:43121 (local address 127.0.0.1:36132)
0924 00:59:40.067136 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:40.067155 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0924 00:59:40.067346 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:59:40.068084 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:59:40.068091 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:59:40.068128 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:59:40.068427 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:40.068435 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:40.070254 (+ 1819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:40.070264 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:40.072396 (+ 2132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:40.072406 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:40.075052 (+ 2646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:40.075059 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:40.075235 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:40.075267 (+ 32us) client_negotiation.cc:769] Sending connection context
0924 00:59:40.075396 (+ 129us) client_negotiation.cc:240] Negotiation successful
0924 00:59:40.075512 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":736,"mutex_wait_us":2,"thread_start_us":543,"threads_started":1}
I20260924 00:59:40.076021 24485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:40.065627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36132 (local address 127.8.64.130:43121)
0924 00:59:40.066194 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:40.066203 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:40.067251 (+ 1048us) server_negotiation.cc:407] Connection header received
0924 00:59:40.067523 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:40.067532 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:40.067791 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:40.067952 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:59:40.068585 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:40.070069 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:40.072545 (+ 2476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:40.073877 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:40.074010 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:40.074206 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:40.075488 (+ 1282us) server_negotiation.cc:299] Negotiation successful
0924 00:59:40.075696 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":330,"threads_started":1}
I20260924 00:59:40.079401 24404 tablet_service.cc:1467] Processing CreateTablet for tablet c31caf0a179f45db836de99e37c3ce3a (DEFAULT_TABLE table=test-table [id=f561c0b94fd340b79945f3e38debfc32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:59:40.079416 24136 tablet_service.cc:1467] Processing CreateTablet for tablet c31caf0a179f45db836de99e37c3ce3a (DEFAULT_TABLE table=test-table [id=f561c0b94fd340b79945f3e38debfc32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:59:40.079418 24270 tablet_service.cc:1467] Processing CreateTablet for tablet c31caf0a179f45db836de99e37c3ce3a (DEFAULT_TABLE table=test-table [id=f561c0b94fd340b79945f3e38debfc32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:59:40.080749 24404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c31caf0a179f45db836de99e37c3ce3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:59:40.080817 24270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c31caf0a179f45db836de99e37c3ce3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:59:40.081585 24136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c31caf0a179f45db836de99e37c3ce3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:59:40.092573 24487 tablet_bootstrap.cc:492] T c31caf0a179f45db836de99e37c3ce3a P 4a1609a2bbb742e4b6722861bd63c4a8: Bootstrap starting.
I20260924 00:59:40.096297 24488 tablet_bootstrap.cc:492] T c31caf0a179f45db836de99e37c3ce3a P ae77c9ce9b674ebbb2f57f0bcc293036: Bootstrap starting.
I20260924 00:59:40.097715 24487 tablet_bootstrap.cc:654] T c31caf0a179f45db836de99e37c3ce3a P 4a1609a2bbb742e4b6722861bd63c4a8: Neither blocks nor log segments found. Creating new log.
I20260924 00:59:40.098512 24489 tablet_bootstrap.cc:492] T c31caf0a179f45db836de99e37c3ce3a P 4727a59d1809474dad8a93ba1bf545be: Bootstrap starting.
I20260924 00:59:40.099175 24487 log.cc:824] T c31caf0a179f45db836de99e37c3ce3a P 4a1609a2bbb742e4b6722861bd63c4a8: Log is configured to *not* fsync() on all Append() calls
I20260924 00:59:40.100075 24488 tablet_bootstrap.cc:654] T c31caf0a179f45db836de99e37c3ce3a P ae77c9ce9b674ebbb2f57f0bcc293036: Neither blocks nor log segments found. Creating new log.
I20260924 00:59:40.101257 24488 log.cc:824] T c31caf0a179f45db836de99e37c3ce3a P ae77c9ce9b674ebbb2f57f0bcc293036: Log is configured to *not* fsync() on all Append() calls
I20260924 00:59:40.101718 24489 tablet_bootstrap.cc:654] T c31caf0a179f45db836de99e37c3ce3a P 4727a59d1809474dad8a93ba1bf545be: Neither blocks nor log segments found. Creating new log.
I20260924 00:59:40.102581 24487 tablet_bootstrap.cc:492] T c31caf0a179f45db836de99e37c3ce3a P 4a1609a2bbb742e4b6722861bd63c4a8: No bootstrap required, opened a new log
I20260924 00:59:40.102900 24487 ts_tablet_manager.cc:1397] T c31caf0a179f45db836de99e37c3ce3a P 4a1609a2bbb742e4b6722861bd63c4a8: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260924 00:59:40.103055 24489 log.cc:824] T c31caf0a179f45db836de99e37c3ce3a P 4727a59d1809474dad8a93ba1bf545be: Log is configured to *not* fsync() on all Append() calls
I20260924 00:59:40.104482 24488 tablet_bootstrap.cc:492] T c31caf0a179f45db836de99e37c3ce3a P ae77c9ce9b674ebbb2f57f0bcc293036: No bootstrap required, opened a new log
I20260924 00:59:40.104761 24488 ts_tablet_manager.cc:1397] T c31caf0a179f45db836de99e37c3ce3a P ae77c9ce9b674ebbb2f57f0bcc293036: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260924 00:59:40.105723 24489 tablet_bootstrap.cc:492] T c31caf0a179f45db836de99e37c3ce3a P 4727a59d1809474dad8a93ba1bf545be: No bootstrap required, opened a new log
I20260924 00:59:40.106432 24489 ts_tablet_manager.cc:1397] T c31caf0a179f45db836de99e37c3ce3a P 4727a59d1809474dad8a93ba1bf545be: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.005s
I20260924 00:59:40.113688 24489 raft_consensus.cc:348] T c31caf0a179f45db836de99e37c3ce3a P 4727a59d1809474dad8a93ba1bf545be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4727a59d1809474dad8a93ba1bf545be" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37443 } } peers { permanent_uuid: "4a1609a2bbb742e4b6722861bd63c4a8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 38205 } } peers { permanent_uuid: "ae77c9ce9b674ebbb2f57f0bcc293036" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 43121 } }
I20260924 00:59:40.113744 24487 raft_consensus.cc:348] T c31caf0a179f45db836de99e37c3ce3a P 4a1609a2bbb742e4b6722861bd63c4a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4727a59d1809474dad8a93ba1bf545be" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37443 } } peers { permanent_uuid: "4a1609a2bbb742e4b6722861bd63c4a8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 38205 } } peers { permanent_uuid: "ae77c9ce9b674ebbb2f57f0bcc293036" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 43121 } }
I20260924 00:59:40.114104 24489 raft_consensus.cc:374] T c31caf0a179f45db836de99e37c3ce3a P 4727a59d1809474dad8a93ba1bf545be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:59:40.114243 24487 raft_consensus.cc:374] T c31caf0a179f45db836de99e37c3ce3a P 4a1609a2bbb742e4b6722861bd63c4a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:59:40.114250 24489 raft_consensus.cc:729] T c31caf0a179f45db836de99e37c3ce3a P 4727a59d1809474dad8a93ba1bf545be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4727a59d1809474dad8a93ba1bf545be, State: Initialized, Role: FOLLOWER
I20260924 00:59:40.114368 24487 raft_consensus.cc:729] T c31caf0a179f45db836de99e37c3ce3a P 4a1609a2bbb742e4b6722861bd63c4a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a1609a2bbb742e4b6722861bd63c4a8, State: Initialized, Role: FOLLOWER
I20260924 00:59:40.114959 24489 consensus_queue.cc:260] T c31caf0a179f45db836de99e37c3ce3a P 4727a59d1809474dad8a93ba1bf545be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4727a59d1809474dad8a93ba1bf545be" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37443 } } peers { permanent_uuid: "4a1609a2bbb742e4b6722861bd63c4a8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 38205 } } peers { permanent_uuid: "ae77c9ce9b674ebbb2f57f0bcc293036" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 43121 } }
I20260924 00:59:40.115075 24487 consensus_queue.cc:260] T c31caf0a179f45db836de99e37c3ce3a P 4a1609a2bbb742e4b6722861bd63c4a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4727a59d1809474dad8a93ba1bf545be" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37443 } } peers { permanent_uuid: "4a1609a2bbb742e4b6722861bd63c4a8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 38205 } } peers { permanent_uuid: "ae77c9ce9b674ebbb2f57f0bcc293036" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 43121 } }
I20260924 00:59:40.115434 24488 raft_consensus.cc:348] T c31caf0a179f45db836de99e37c3ce3a P ae77c9ce9b674ebbb2f57f0bcc293036 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4727a59d1809474dad8a93ba1bf545be" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37443 } } peers { permanent_uuid: "4a1609a2bbb742e4b6722861bd63c4a8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 38205 } } peers { permanent_uuid: "ae77c9ce9b674ebbb2f57f0bcc293036" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 43121 } }
I20260924 00:59:40.115960 24488 raft_consensus.cc:374] T c31caf0a179f45db836de99e37c3ce3a P ae77c9ce9b674ebbb2f57f0bcc293036 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:59:40.116109 24488 raft_consensus.cc:729] T c31caf0a179f45db836de99e37c3ce3a P ae77c9ce9b674ebbb2f57f0bcc293036 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae77c9ce9b674ebbb2f57f0bcc293036, State: Initialized, Role: FOLLOWER
I20260924 00:59:40.116900 24488 consensus_queue.cc:260] T c31caf0a179f45db836de99e37c3ce3a P ae77c9ce9b674ebbb2f57f0bcc293036 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4727a59d1809474dad8a93ba1bf545be" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37443 } } peers { permanent_uuid: "4a1609a2bbb742e4b6722861bd63c4a8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 38205 } } peers { permanent_uuid: "ae77c9ce9b674ebbb2f57f0bcc293036" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 43121 } }
I20260924 00:59:40.117180 24487 ts_tablet_manager.cc:1428] T c31caf0a179f45db836de99e37c3ce3a P 4a1609a2bbb742e4b6722861bd63c4a8: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260924 00:59:40.118357 24469 heartbeater.cc:503] Master 127.8.64.190:42983 was elected leader, sending a full tablet report...
I20260924 00:59:40.119051 24335 heartbeater.cc:503] Master 127.8.64.190:42983 was elected leader, sending a full tablet report...
I20260924 00:59:40.119606 24489 ts_tablet_manager.cc:1428] T c31caf0a179f45db836de99e37c3ce3a P 4727a59d1809474dad8a93ba1bf545be: Time spent starting tablet: real 0.013s user 0.005s sys 0.007s
I20260924 00:59:40.120682 24488 ts_tablet_manager.cc:1428] T c31caf0a179f45db836de99e37c3ce3a P ae77c9ce9b674ebbb2f57f0bcc293036: Time spent starting tablet: real 0.016s user 0.015s sys 0.000s
I20260924 00:59:40.185220 24495 raft_consensus.cc:482] T c31caf0a179f45db836de99e37c3ce3a P ae77c9ce9b674ebbb2f57f0bcc293036 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:59:40.185542 24495 raft_consensus.cc:504] T c31caf0a179f45db836de99e37c3ce3a P ae77c9ce9b674ebbb2f57f0bcc293036 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4727a59d1809474dad8a93ba1bf545be" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37443 } } peers { permanent_uuid: "4a1609a2bbb742e4b6722861bd63c4a8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 38205 } } peers { permanent_uuid: "ae77c9ce9b674ebbb2f57f0bcc293036" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 43121 } }
I20260924 00:59:40.187366 24495 leader_election.cc:288] T c31caf0a179f45db836de99e37c3ce3a P ae77c9ce9b674ebbb2f57f0bcc293036 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4727a59d1809474dad8a93ba1bf545be (127.8.64.129:37443), 4a1609a2bbb742e4b6722861bd63c4a8 (127.8.64.131:38205)
W20260924 00:59:40.189949 24202 tablet.cc:2369] T c31caf0a179f45db836de99e37c3ce3a P 4727a59d1809474dad8a93ba1bf545be: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:59:40.195006 24484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:40.187878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:34655 (local address 127.8.64.131:38205)
0924 00:59:40.188229 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:40.188238 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:40.189038 (+ 800us) server_negotiation.cc:407] Connection header received
0924 00:59:40.189197 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:40.189203 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:40.189279 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:40.189378 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:59:40.190124 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:40.191157 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:40.193155 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:40.193930 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:40.194087 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:40.194174 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:40.194598 (+ 424us) server_negotiation.cc:299] Negotiation successful
0924 00:59:40.194732 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149}
I20260924 00:59:40.195016 24482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:40.187823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:43223 (local address 127.8.64.129:37443)
0924 00:59:40.188060 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:40.188066 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:40.188840 (+ 774us) server_negotiation.cc:407] Connection header received
0924 00:59:40.189026 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:40.189033 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:40.189119 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:40.189207 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:59:40.189901 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:40.191040 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:40.193193 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:40.193954 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:40.194150 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:40.194219 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:40.194642 (+ 423us) server_negotiation.cc:299] Negotiation successful
0924 00:59:40.194764 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260924 00:59:40.195175 24497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:40.187953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.131:38205 (local address 127.8.64.130:34655)
0924 00:59:40.188862 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:40.188886 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0924 00:59:40.189048 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:59:40.189404 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:59:40.189410 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:59:40.189438 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:59:40.189913 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:40.189930 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:40.191331 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:40.191335 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:40.192994 (+ 1659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:40.193006 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:40.194167 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:40.194174 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:40.194403 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:40.194464 (+ 61us) client_negotiation.cc:769] Sending connection context
0924 00:59:40.194574 (+ 110us) client_negotiation.cc:240] Negotiation successful
0924 00:59:40.194698 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":765,"mutex_wait_us":95,"thread_start_us":581,"threads_started":1}
I20260924 00:59:40.195209 24496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:40.187723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:37443 (local address 127.8.64.130:43223)
0924 00:59:40.188696 (+ 973us) negotiation.cc:107] Waiting for socket to connect
0924 00:59:40.188717 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0924 00:59:40.188874 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:59:40.189238 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:59:40.189243 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:59:40.189275 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:59:40.189726 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:40.189737 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:40.191215 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:40.191224 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:40.193007 (+ 1783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:59:40.193018 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:59:40.194205 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:59:40.194211 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:59:40.194474 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:40.194498 (+ 24us) client_negotiation.cc:769] Sending connection context
0924 00:59:40.194611 (+ 113us) client_negotiation.cc:240] Negotiation successful
0924 00:59:40.194730 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":757,"mutex_wait_us":163,"spinlock_wait_cycles":22272,"thread_start_us":516,"threads_started":1}
I20260924 00:59:40.196377 24424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c31caf0a179f45db836de99e37c3ce3a" candidate_uuid: "ae77c9ce9b674ebbb2f57f0bcc293036" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a1609a2bbb742e4b6722861bd63c4a8" is_pre_election: true
I20260924 00:59:40.196376 24156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c31caf0a179f45db836de99e37c3ce3a" candidate_uuid: "ae77c9ce9b674ebbb2f57f0bcc293036" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4727a59d1809474dad8a93ba1bf545be" is_pre_election: true
I20260924 00:59:40.196861 24156 raft_consensus.cc:2454] T c31caf0a179f45db836de99e37c3ce3a P 4727a59d1809474dad8a93ba1bf545be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae77c9ce9b674ebbb2f57f0bcc293036 in term 0.
I20260924 00:59:40.196859 24424 raft_consensus.cc:2454] T c31caf0a179f45db836de99e37c3ce3a P 4a1609a2bbb742e4b6722861bd63c4a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae77c9ce9b674ebbb2f57f0bcc293036 in term 0.
I20260924 00:59:40.197522 24224 leader_election.cc:302] T c31caf0a179f45db836de99e37c3ce3a P ae77c9ce9b674ebbb2f57f0bcc293036 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4727a59d1809474dad8a93ba1bf545be, 4a1609a2bbb742e4b6722861bd63c4a8, ae77c9ce9b674ebbb2f57f0bcc293036; no voters:
I20260924 00:59:40.197935 24495 raft_consensus.cc:2781] T c31caf0a179f45db836de99e37c3ce3a P ae77c9ce9b674ebbb2f57f0bcc293036 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:59:40.198084 24495 raft_consensus.cc:482] T c31caf0a179f45db836de99e37c3ce3a P ae77c9ce9b674ebbb2f57f0bcc293036 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:59:40.198212 24495 raft_consensus.cc:3037] T c31caf0a179f45db836de99e37c3ce3a P ae77c9ce9b674ebbb2f57f0bcc293036 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:59:40.200410 24495 raft_consensus.cc:504] T c31caf0a179f45db836de99e37c3ce3a P ae77c9ce9b674ebbb2f57f0bcc293036 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4727a59d1809474dad8a93ba1bf545be" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37443 } } peers { permanent_uuid: "4a1609a2bbb742e4b6722861bd63c4a8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 38205 } } peers { permanent_uuid: "ae77c9ce9b674ebbb2f57f0bcc293036" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 43121 } }
I20260924 00:59:40.201071 24495 leader_election.cc:288] T c31caf0a179f45db836de99e37c3ce3a P ae77c9ce9b674ebbb2f57f0bcc293036 [CANDIDATE]: Term 1 election: Requested vote from peers 4727a59d1809474dad8a93ba1bf545be (127.8.64.129:37443), 4a1609a2bbb742e4b6722861bd63c4a8 (127.8.64.131:38205)
I20260924 00:59:40.201429 24156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c31caf0a179f45db836de99e37c3ce3a" candidate_uuid: "ae77c9ce9b674ebbb2f57f0bcc293036" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4727a59d1809474dad8a93ba1bf545be"
I20260924 00:59:40.201647 24156 raft_consensus.cc:3037] T c31caf0a179f45db836de99e37c3ce3a P 4727a59d1809474dad8a93ba1bf545be [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:59:40.201576 24424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c31caf0a179f45db836de99e37c3ce3a" candidate_uuid: "ae77c9ce9b674ebbb2f57f0bcc293036" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a1609a2bbb742e4b6722861bd63c4a8"
I20260924 00:59:40.201828 24424 raft_consensus.cc:3037] T c31caf0a179f45db836de99e37c3ce3a P 4a1609a2bbb742e4b6722861bd63c4a8 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:59:40.204406 24156 raft_consensus.cc:2454] T c31caf0a179f45db836de99e37c3ce3a P 4727a59d1809474dad8a93ba1bf545be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae77c9ce9b674ebbb2f57f0bcc293036 in term 1.
I20260924 00:59:40.204733 24424 raft_consensus.cc:2454] T c31caf0a179f45db836de99e37c3ce3a P 4a1609a2bbb742e4b6722861bd63c4a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae77c9ce9b674ebbb2f57f0bcc293036 in term 1.
I20260924 00:59:40.204886 24226 leader_election.cc:302] T c31caf0a179f45db836de99e37c3ce3a P ae77c9ce9b674ebbb2f57f0bcc293036 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4727a59d1809474dad8a93ba1bf545be, ae77c9ce9b674ebbb2f57f0bcc293036; no voters:
I20260924 00:59:40.205215 24495 raft_consensus.cc:2781] T c31caf0a179f45db836de99e37c3ce3a P ae77c9ce9b674ebbb2f57f0bcc293036 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:59:40.206568 24495 raft_consensus.cc:686] T c31caf0a179f45db836de99e37c3ce3a P ae77c9ce9b674ebbb2f57f0bcc293036 [term 1 LEADER]: Becoming Leader. State: Replica: ae77c9ce9b674ebbb2f57f0bcc293036, State: Running, Role: LEADER
I20260924 00:59:40.207041 24495 consensus_queue.cc:237] T c31caf0a179f45db836de99e37c3ce3a P ae77c9ce9b674ebbb2f57f0bcc293036 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4727a59d1809474dad8a93ba1bf545be" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37443 } } peers { permanent_uuid: "4a1609a2bbb742e4b6722861bd63c4a8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 38205 } } peers { permanent_uuid: "ae77c9ce9b674ebbb2f57f0bcc293036" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 43121 } }
W20260924 00:59:40.210882 24336 tablet.cc:2369] T c31caf0a179f45db836de99e37c3ce3a P ae77c9ce9b674ebbb2f57f0bcc293036: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:59:40.211908 24012 catalog_manager.cc:5495] T c31caf0a179f45db836de99e37c3ce3a P ae77c9ce9b674ebbb2f57f0bcc293036 reported cstate change: term changed from 0 to 1, leader changed from <none> to ae77c9ce9b674ebbb2f57f0bcc293036 (127.8.64.130). New cstate: current_term: 1 leader_uuid: "ae77c9ce9b674ebbb2f57f0bcc293036" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4727a59d1809474dad8a93ba1bf545be" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a1609a2bbb742e4b6722861bd63c4a8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 38205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae77c9ce9b674ebbb2f57f0bcc293036" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 43121 } health_report { overall_health: HEALTHY } } }
W20260924 00:59:40.228483 24470 tablet.cc:2369] T c31caf0a179f45db836de99e37c3ce3a P 4a1609a2bbb742e4b6722861bd63c4a8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:59:40.232752 24485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:40.227147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36136 (local address 127.8.64.130:43121)
0924 00:59:40.227349 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:40.227355 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:40.227460 (+ 105us) server_negotiation.cc:407] Connection header received
0924 00:59:40.227742 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:40.227746 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:40.227824 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:40.227977 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:59:40.228376 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:40.229149 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:40.230678 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:40.231071 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:40.231218 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:40.231679 (+ 461us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:59:40.232159 (+ 480us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:59:40.232331 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:40.232389 (+ 58us) server_negotiation.cc:299] Negotiation successful
0924 00:59:40.232468 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260924 00:59:40.240870 24424 raft_consensus.cc:1261] T c31caf0a179f45db836de99e37c3ce3a P 4a1609a2bbb742e4b6722861bd63c4a8 [term 1 FOLLOWER]: Refusing update from remote peer ae77c9ce9b674ebbb2f57f0bcc293036: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:59:40.240870 24156 raft_consensus.cc:1261] T c31caf0a179f45db836de99e37c3ce3a P 4727a59d1809474dad8a93ba1bf545be [term 1 FOLLOWER]: Refusing update from remote peer ae77c9ce9b674ebbb2f57f0bcc293036: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:59:40.241909 24498 consensus_queue.cc:1035] T c31caf0a179f45db836de99e37c3ce3a P ae77c9ce9b674ebbb2f57f0bcc293036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a1609a2bbb742e4b6722861bd63c4a8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 38205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:59:40.242262 24495 consensus_queue.cc:1035] T c31caf0a179f45db836de99e37c3ce3a P ae77c9ce9b674ebbb2f57f0bcc293036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4727a59d1809474dad8a93ba1bf545be" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:59:40.253557 24501 mvcc.cc:205] Tried to move back new op lower bound from 7332706632656363520 to 7332706632531644416. Current Snapshot: MvccSnapshot[applied={T|T < 7332706632656363520}]
I20260924 00:59:40.255532 24503 mvcc.cc:205] Tried to move back new op lower bound from 7332706632656363520 to 7332706632531644416. Current Snapshot: MvccSnapshot[applied={T|T < 7332706632656363520}]
I20260924 00:59:40.269090 24010 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60360:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:59:40.269347 24010 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60360:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:59:40.273113 24010 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770: Sending DeleteTablet for 3 replicas of tablet c31caf0a179f45db836de99e37c3ce3a
I20260924 00:59:40.274456 24404 tablet_service.cc:1514] Processing DeleteTablet for tablet c31caf0a179f45db836de99e37c3ce3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:59:40 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:55196
I20260924 00:59:40.274456 24136 tablet_service.cc:1514] Processing DeleteTablet for tablet c31caf0a179f45db836de99e37c3ce3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:59:40 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:43106
I20260924 00:59:40.274700 24270 tablet_service.cc:1514] Processing DeleteTablet for tablet c31caf0a179f45db836de99e37c3ce3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:59:40 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:36132
I20260924 00:59:40.275915 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 24073
I20260924 00:59:40.285097 24512 tablet_replica.cc:330] T c31caf0a179f45db836de99e37c3ce3a P 4727a59d1809474dad8a93ba1bf545be: stopping tablet replica
I20260924 00:59:40.285702 24512 raft_consensus.cc:2229] T c31caf0a179f45db836de99e37c3ce3a P 4727a59d1809474dad8a93ba1bf545be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:59:40.286343 24512 raft_consensus.cc:2258] T c31caf0a179f45db836de99e37c3ce3a P 4727a59d1809474dad8a93ba1bf545be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:59:40.286940 24514 tablet_replica.cc:330] T c31caf0a179f45db836de99e37c3ce3a P ae77c9ce9b674ebbb2f57f0bcc293036: stopping tablet replica
I20260924 00:59:40.287673 24514 raft_consensus.cc:2229] T c31caf0a179f45db836de99e37c3ce3a P ae77c9ce9b674ebbb2f57f0bcc293036 [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:59:40.287910 24513 tablet_replica.cc:330] T c31caf0a179f45db836de99e37c3ce3a P 4a1609a2bbb742e4b6722861bd63c4a8: stopping tablet replica
I20260924 00:59:40.288348 24513 raft_consensus.cc:2229] T c31caf0a179f45db836de99e37c3ce3a P 4a1609a2bbb742e4b6722861bd63c4a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:59:40.288403 24514 raft_consensus.cc:2258] T c31caf0a179f45db836de99e37c3ce3a P ae77c9ce9b674ebbb2f57f0bcc293036 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:59:40.288875 24513 raft_consensus.cc:2258] T c31caf0a179f45db836de99e37c3ce3a P 4a1609a2bbb742e4b6722861bd63c4a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:59:40.290750 24514 ts_tablet_manager.cc:1905] T c31caf0a179f45db836de99e37c3ce3a P ae77c9ce9b674ebbb2f57f0bcc293036: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:59:40.291077 24513 ts_tablet_manager.cc:1905] T c31caf0a179f45db836de99e37c3ce3a P 4a1609a2bbb742e4b6722861bd63c4a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:59:40.291852 24512 ts_tablet_manager.cc:1905] T c31caf0a179f45db836de99e37c3ce3a P 4727a59d1809474dad8a93ba1bf545be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:59:40.298477 24514 ts_tablet_manager.cc:1918] T c31caf0a179f45db836de99e37c3ce3a P ae77c9ce9b674ebbb2f57f0bcc293036: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:59:40.298727 24514 log.cc:1196] T c31caf0a179f45db836de99e37c3ce3a P ae77c9ce9b674ebbb2f57f0bcc293036: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-1/wal/wals/c31caf0a179f45db836de99e37c3ce3a
I20260924 00:59:40.299342 24514 ts_tablet_manager.cc:1939] T c31caf0a179f45db836de99e37c3ce3a P ae77c9ce9b674ebbb2f57f0bcc293036: Deleting consensus metadata
I20260924 00:59:40.300719 24512 ts_tablet_manager.cc:1918] T c31caf0a179f45db836de99e37c3ce3a P 4727a59d1809474dad8a93ba1bf545be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:59:40.300869 24512 log.cc:1196] T c31caf0a179f45db836de99e37c3ce3a P 4727a59d1809474dad8a93ba1bf545be: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-0/wal/wals/c31caf0a179f45db836de99e37c3ce3a
I20260924 00:59:40.301319 24513 ts_tablet_manager.cc:1918] T c31caf0a179f45db836de99e37c3ce3a P 4a1609a2bbb742e4b6722861bd63c4a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:59:40.301425 24512 ts_tablet_manager.cc:1939] T c31caf0a179f45db836de99e37c3ce3a P 4727a59d1809474dad8a93ba1bf545be: Deleting consensus metadata
I20260924 00:59:40.301415 23997 catalog_manager.cc:4841] TS ae77c9ce9b674ebbb2f57f0bcc293036 (127.8.64.130:43121): tablet c31caf0a179f45db836de99e37c3ce3a (table test-table [id=f561c0b94fd340b79945f3e38debfc32]) successfully deleted
I20260924 00:59:40.301579 24513 log.cc:1196] T c31caf0a179f45db836de99e37c3ce3a P 4a1609a2bbb742e4b6722861bd63c4a8: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-2/wal/wals/c31caf0a179f45db836de99e37c3ce3a
I20260924 00:59:40.302066 24513 ts_tablet_manager.cc:1939] T c31caf0a179f45db836de99e37c3ce3a P 4a1609a2bbb742e4b6722861bd63c4a8: Deleting consensus metadata
I20260924 00:59:40.303267 23999 catalog_manager.cc:4841] TS 4727a59d1809474dad8a93ba1bf545be (127.8.64.129:37443): tablet c31caf0a179f45db836de99e37c3ce3a (table test-table [id=f561c0b94fd340b79945f3e38debfc32]) successfully deleted
I20260924 00:59:40.303695 23997 catalog_manager.cc:4841] TS 4a1609a2bbb742e4b6722861bd63c4a8 (127.8.64.131:38205): tablet c31caf0a179f45db836de99e37c3ce3a (table test-table [id=f561c0b94fd340b79945f3e38debfc32]) successfully deleted
Sep 24 00:59:40 dist-test-slave-mmhs krb5kdc[23967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211577, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.129@KRBTEST.COM
I20260924 00:59:40.313905 24482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:40.277280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43112 (local address 127.8.64.129:37443)
0924 00:59:40.277506 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:40.277516 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:40.287311 (+ 9795us) server_negotiation.cc:407] Connection header received
0924 00:59:40.287578 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:40.287586 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:40.287694 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:40.287799 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:40.291428 (+ 3629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:40.292557 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:40.294755 (+ 2198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:40.295374 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:40.295472 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:40.306956 (+ 11484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:40.306971 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:40.306991 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:40.307036 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:40.310801 (+ 3765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:40.311594 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:40.311604 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:40.311607 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:40.311784 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:40.312155 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:40.312160 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:40.312163 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:40.312625 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:40.312933 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:40.313254 (+ 321us) server_negotiation.cc:299] Negotiation successful
0924 00:59:40.313478 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260924 00:59:40.314632 24196 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:59:40.407223 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 24073
I20260924 00:59:40.431270 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 24207
Sep 24 00:59:40 dist-test-slave-mmhs krb5kdc[23967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211577, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.130@KRBTEST.COM
I20260924 00:59:40.448988 24485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:40.432046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36152 (local address 127.8.64.130:43121)
0924 00:59:40.432228 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:40.432232 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:40.432321 (+ 89us) server_negotiation.cc:407] Connection header received
0924 00:59:40.432513 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:40.432517 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:40.432580 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:40.432691 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:40.434508 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:40.435346 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:40.437370 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:40.437788 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:40.437879 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:40.443007 (+ 5128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:40.443019 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:40.443033 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:40.443068 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:40.446069 (+ 3001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:40.446863 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:40.446870 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:40.446872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:40.447018 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:40.447422 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:40.447426 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:40.447428 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:40.447783 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:40.448111 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:40.448417 (+ 306us) server_negotiation.cc:299] Negotiation successful
0924 00:59:40.448633 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260924 00:59:40.449642 24330 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:59:40.563578 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 24207
I20260924 00:59:40.582676 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 24341
Sep 24 00:59:40 dist-test-slave-mmhs krb5kdc[23967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211577, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.131@KRBTEST.COM
I20260924 00:59:40.597802 24484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:59:40.583483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55202 (local address 127.8.64.131:38205)
0924 00:59:40.583688 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0924 00:59:40.583694 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:59:40.583714 (+ 20us) server_negotiation.cc:407] Connection header received
0924 00:59:40.583937 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:59:40.583947 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:59:40.584023 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:59:40.584137 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:59:40.585556 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:40.586338 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:40.587671 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:59:40.588090 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:59:40.588235 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:59:40.592712 (+ 4477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:59:40.592726 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:59:40.592743 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:59:40.592776 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:59:40.595409 (+ 2633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:40.596064 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:40.596071 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:40.596073 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:40.596221 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:59:40.596538 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:59:40.596541 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:59:40.596543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:59:40.596890 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:59:40.597097 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0924 00:59:40.597365 (+ 268us) server_negotiation.cc:299] Negotiation successful
0924 00:59:40.597530 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260924 00:59:40.598538 24464 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:59:40.688778 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 24341
I20260924 00:59:40.705986 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 23980
I20260924 00:59:40.707265 24042 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 00:59:40.817124 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 23980
I20260924 01:00:00.838884 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:42983
--webserver_interface=127.8.64.190
--webserver_port=34309
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:35317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:42983
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 01:00:01.166221 24520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:00:01.166623 24520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:00:01.166783 24520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:00:01.173895 24520 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260924 01:00:01.174031 24520 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260924 01:00:01.174103 24520 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260924 01:00:01.174181 24520 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 01:00:01.174259 24520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:00:01.174393 24520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:00:01.174481 24520 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 01:00:01.174576 24520 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 01:00:01.184262 24520 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:35317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:42983
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:42983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=34309
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:00:01.185554 24520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:00:01.186914 24520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:00:01.214375 24526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:01.214375 24529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:01.214385 24527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:00:01.220130 24520 server_base.cc:1034] running on GCE node
I20260924 01:00:01.221077 24520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:00:01.222895 24520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:00:01.224159 24520 hybrid_clock.cc:648] HybridClock initialized: now 1790211601224097 us; error 63 us; skew 500 ppm
I20260924 01:00:01.224566 24520 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 01:00:01 dist-test-slave-mmhs krb5kdc[23967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211601, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 01:00:01.232702 24520 init.cc:377] Logged in from keytab as kudu/127.8.64.190@KRBTEST.COM (short username kudu)
I20260924 01:00:01.234601 24520 webserver.cc:460] Webserver started at http://127.8.64.190:34309/ using document root <none> and password file <none>
I20260924 01:00:01.235420 24520 fs_manager.cc:359] Metadata directory not provided
I20260924 01:00:01.235576 24520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:00:01.241762 24520 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.002s
I20260924 01:00:01.245013 24536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:01.246692 24520 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260924 01:00:01.246909 24520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "1ffc1d56a39440f09aa42a5e0174c770"
format_stamp: "Formatted at 2026-09-24 00:59:38 on dist-test-slave-mmhs"
server_key: "04e8f38f2e0aaffd2381c2ef1e98b816"
server_key_iv: "6ecfc60af2ee8cc0642ed76bbe6f88ce"
server_key_version: "encryptionkey@0"
I20260924 01:00:01.248114 24520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:00:01.280017 24520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:00:01.284672 24520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:00:01.284976 24520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:00:01.304931 24520 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.190:42983
I20260924 01:00:01.304965 24587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.190:42983 every 8 connection(s)
I20260924 01:00:01.306579 24520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 01:00:01.313786 24588 sys_catalog.cc:263] Verifying existing consensus state
I20260924 01:00:01.314716 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 24520
I20260924 01:00:01.315676 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.129:37443
--local_ip_for_outbound_sockets=127.8.64.129
--tserver_master_addrs=127.8.64.190:42983
--webserver_port=36865
--webserver_interface=127.8.64.129
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab
--principal=kudu/127.8.64.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:35317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 01:00:01.318795 24588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770: Bootstrap starting.
I20260924 01:00:01.340577 24588 log.cc:824] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770: Log is configured to *not* fsync() on all Append() calls
I20260924 01:00:01.352901 24588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 01:00:01.353431 24588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770: Bootstrap complete.
I20260924 01:00:01.362618 24588 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ffc1d56a39440f09aa42a5e0174c770" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42983 } }
I20260924 01:00:01.363411 24588 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ffc1d56a39440f09aa42a5e0174c770, State: Initialized, Role: FOLLOWER
I20260924 01:00:01.364046 24588 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ffc1d56a39440f09aa42a5e0174c770" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42983 } }
I20260924 01:00:01.364288 24588 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 01:00:01.364413 24588 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 01:00:01.364630 24588 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [term 1 FOLLOWER]: Advancing to term 2
I20260924 01:00:01.368505 24588 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ffc1d56a39440f09aa42a5e0174c770" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42983 } }
I20260924 01:00:01.369024 24588 leader_election.cc:302] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ffc1d56a39440f09aa42a5e0174c770; no voters:
I20260924 01:00:01.369655 24588 leader_election.cc:288] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260924 01:00:01.369805 24592 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 01:00:01.371182 24592 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [term 2 LEADER]: Becoming Leader. State: Replica: 1ffc1d56a39440f09aa42a5e0174c770, State: Running, Role: LEADER
I20260924 01:00:01.371717 24592 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ffc1d56a39440f09aa42a5e0174c770" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42983 } }
I20260924 01:00:01.372102 24588 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 01:00:01.374257 24593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1ffc1d56a39440f09aa42a5e0174c770" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ffc1d56a39440f09aa42a5e0174c770" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42983 } } }
I20260924 01:00:01.375255 24593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [sys.catalog]: This master's current role is: LEADER
I20260924 01:00:01.374845 24594 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ffc1d56a39440f09aa42a5e0174c770. Latest consensus state: current_term: 2 leader_uuid: "1ffc1d56a39440f09aa42a5e0174c770" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ffc1d56a39440f09aa42a5e0174c770" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42983 } } }
I20260924 01:00:01.375996 24594 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [sys.catalog]: This master's current role is: LEADER
I20260924 01:00:01.385303 24598 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 01:00:01.394690 24598 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 01:00:01.396512 24598 catalog_manager.cc:1259] Loaded cluster ID: 6318508f6afc4040b7443bf314edebcd
I20260924 01:00:01.396634 24598 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 01:00:01.401684 24598 catalog_manager.cc:1504] Loading token signing keys...
I20260924 01:00:01.403664 24598 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770: Loaded TSK: 0
I20260924 01:00:01.427139 24609 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260924 01:00:01.440907 24598 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770: Generated new TSK 1
I20260924 01:00:01.441301 24598 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260924 01:00:01.697414 24590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:00:01.697754 24590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:00:01.697940 24590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:00:01.705394 24590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:00:01.705572 24590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:00:01.705756 24590 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.129
I20260924 01:00:01.715978 24590 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:35317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab
--principal=kudu/127.8.64.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.129:37443
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.64.129
--webserver_port=36865
--tserver_master_addrs=127.8.64.190:42983
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.129
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:00:01.717386 24590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:00:01.718868 24590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:00:01.734632 24616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:01.734797 24615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:00:01.735260 24590 server_base.cc:1034] running on GCE node
W20260924 01:00:01.734829 24618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:00:01.736135 24590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:00:01.737267 24590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:00:01.738503 24590 hybrid_clock.cc:648] HybridClock initialized: now 1790211601738426 us; error 68 us; skew 500 ppm
I20260924 01:00:01.738857 24590 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 01:00:01 dist-test-slave-mmhs krb5kdc[23967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211601, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 01:00:01.742621 24590 init.cc:377] Logged in from keytab as kudu/127.8.64.129@KRBTEST.COM (short username kudu)
I20260924 01:00:01.744187 24590 webserver.cc:460] Webserver started at http://127.8.64.129:36865/ using document root <none> and password file <none>
I20260924 01:00:01.744913 24590 fs_manager.cc:359] Metadata directory not provided
I20260924 01:00:01.745059 24590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:00:01.750803 24590 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260924 01:00:01.754002 24625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:01.755501 24590 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 01:00:01.755705 24590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-0/wal
uuid: "4727a59d1809474dad8a93ba1bf545be"
format_stamp: "Formatted at 2026-09-24 00:59:38 on dist-test-slave-mmhs"
server_key: "95872cadad7a1c8ac90e941ca2919790"
server_key_iv: "40db947aad7907bb11b9f1d79ce224b1"
server_key_version: "encryptionkey@0"
I20260924 01:00:01.756711 24590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:00:01.783816 24590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:00:01.787840 24590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:00:01.788156 24590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:00:01.789124 24590 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 01:00:01.790465 24590 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 01:00:01.790603 24590 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:01.790722 24590 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 01:00:01.790819 24590 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:01.824445 24590 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.129:37443
I20260924 01:00:01.824532 24737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.129:37443 every 8 connection(s)
I20260924 01:00:01.825924 24590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
I20260924 01:00:01.834111 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 24590
I20260924 01:00:01.834834 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.130:43121
--local_ip_for_outbound_sockets=127.8.64.130
--tserver_master_addrs=127.8.64.190:42983
--webserver_port=33007
--webserver_interface=127.8.64.130
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab
--principal=kudu/127.8.64.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:35317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 01:00:01 dist-test-slave-mmhs krb5kdc[23967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211601, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 01:00:01.851348 24741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:01.829044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.129:57589)
0924 01:00:01.829872 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:01.829907 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:01.831720 (+ 1813us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:01.833099 (+ 1379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:01.833110 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:01.833840 (+ 730us) client_negotiation.cc:189] Negotiated authn=SASL
0924 01:00:01.834475 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:01.834495 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:01.836323 (+ 1828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:01.836336 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:01.837588 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:01.837599 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:01.838730 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:01.838737 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:01.839025 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:01.839779 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 01:00:01.839803 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0924 01:00:01.842251 (+ 2448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 01:00:01.846911 (+ 4660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:00:01.846919 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:00:01.846932 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:00:01.847416 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:00:01.847942 (+ 526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:00:01.847946 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:00:01.847949 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:00:01.848163 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:00:01.849192 (+ 1029us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 01:00:01.849198 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 01:00:01.849512 (+ 314us) client_negotiation.cc:769] Sending connection context
0924 01:00:01.849742 (+ 230us) client_negotiation.cc:240] Negotiation successful
0924 01:00:01.850148 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":322,"threads_started":1}
I20260924 01:00:01.851938 24740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:01.828886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:57589 (local address 127.8.64.190:42983)
0924 01:00:01.829611 (+ 725us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:01.829628 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:01.830047 (+ 419us) server_negotiation.cc:407] Connection header received
0924 01:00:01.832256 (+ 2209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:01.832268 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:01.832593 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:01.832936 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:01.834709 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:01.836154 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:01.837740 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:01.838480 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:01.838743 (+ 263us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:01.842641 (+ 3898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:01.842649 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:01.842665 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:00:01.842701 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:01.846720 (+ 4019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:01.847578 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:01.847589 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:01.847599 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:01.847804 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:01.848293 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:01.848301 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:01.848307 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:01.849018 (+ 711us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:01.849241 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:01.850015 (+ 774us) server_negotiation.cc:299] Negotiation successful
0924 01:00:01.850546 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":299,"threads_started":1}
I20260924 01:00:01.854919 24738 heartbeater.cc:347] Connected to a master server at 127.8.64.190:42983
I20260924 01:00:01.855273 24738 heartbeater.cc:464] Registering TS with master...
I20260924 01:00:01.856237 24738 heartbeater.cc:511] Master 127.8.64.190:42983 requested a full tablet report, sending...
I20260924 01:00:01.858779 24553 ts_manager.cc:194] Registered new tserver with Master: 4727a59d1809474dad8a93ba1bf545be (127.8.64.129:37443)
I20260924 01:00:01.860281 24553 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.129@KRBTEST.COM'} at 127.8.64.129:57589
W20260924 01:00:02.164695 24742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:00:02.164981 24742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:00:02.165149 24742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:00:02.172415 24742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:00:02.172549 24742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:00:02.172756 24742 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.130
I20260924 01:00:02.182446 24742 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:35317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab
--principal=kudu/127.8.64.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.130:43121
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.64.130
--webserver_port=33007
--tserver_master_addrs=127.8.64.190:42983
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.130
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:00:02.183770 24742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:00:02.185077 24742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:00:02.198269 24748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:02.198371 24747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:02.198316 24750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:00:02.199149 24742 server_base.cc:1034] running on GCE node
I20260924 01:00:02.199738 24742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:00:02.200645 24742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:00:02.201951 24742 hybrid_clock.cc:648] HybridClock initialized: now 1790211602201909 us; error 54 us; skew 500 ppm
I20260924 01:00:02.202354 24742 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 01:00:02 dist-test-slave-mmhs krb5kdc[23967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211602, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 01:00:02.205439 24742 init.cc:377] Logged in from keytab as kudu/127.8.64.130@KRBTEST.COM (short username kudu)
I20260924 01:00:02.206996 24742 webserver.cc:460] Webserver started at http://127.8.64.130:33007/ using document root <none> and password file <none>
I20260924 01:00:02.207676 24742 fs_manager.cc:359] Metadata directory not provided
I20260924 01:00:02.207831 24742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:00:02.213171 24742 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260924 01:00:02.215822 24757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:02.217109 24742 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260924 01:00:02.217279 24742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-1/wal
uuid: "ae77c9ce9b674ebbb2f57f0bcc293036"
format_stamp: "Formatted at 2026-09-24 00:59:39 on dist-test-slave-mmhs"
server_key: "7ee3772027126ed8f50e0a5594458d7b"
server_key_iv: "867be1ef8dde9ee0daf38d095f63ce6c"
server_key_version: "encryptionkey@0"
I20260924 01:00:02.218221 24742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:00:02.239809 24742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:00:02.244068 24742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:00:02.244345 24742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:00:02.245260 24742 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 01:00:02.246623 24742 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 01:00:02.246747 24742 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:02.246881 24742 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 01:00:02.246969 24742 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:02.278412 24742 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.130:43121
I20260924 01:00:02.278455 24869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.130:43121 every 8 connection(s)
I20260924 01:00:02.279829 24742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
I20260924 01:00:02.283500 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 24742
I20260924 01:00:02.284256 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.131:38205
--local_ip_for_outbound_sockets=127.8.64.131
--tserver_master_addrs=127.8.64.190:42983
--webserver_port=43801
--webserver_interface=127.8.64.131
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab
--principal=kudu/127.8.64.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:35317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 01:00:02 dist-test-slave-mmhs krb5kdc[23967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211602, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.130@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 01:00:02.302906 24740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:02.282738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:42205 (local address 127.8.64.190:42983)
0924 01:00:02.282947 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:02.282953 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:02.284797 (+ 1844us) server_negotiation.cc:407] Connection header received
0924 01:00:02.287783 (+ 2986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:02.287788 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:02.287859 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:02.287982 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:02.290001 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:02.290777 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:02.293186 (+ 2409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:02.293624 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:02.293730 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:02.297336 (+ 3606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:02.297341 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:02.297344 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:00:02.297374 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:02.299995 (+ 2621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:02.300683 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:02.300689 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:02.300694 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:02.300811 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:02.301229 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:02.301234 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:02.301237 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:02.301505 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:02.301657 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:02.302320 (+ 663us) server_negotiation.cc:299] Negotiation successful
0924 01:00:02.302563 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260924 01:00:02.303875 24872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:02.283283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.130:42205)
0924 01:00:02.284642 (+ 1359us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:02.284687 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:02.287485 (+ 2798us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:02.288233 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:02.288247 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:02.289014 (+ 767us) client_negotiation.cc:189] Negotiated authn=SASL
0924 01:00:02.289817 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:02.289847 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:02.290933 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:02.290941 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:02.293020 (+ 2079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:02.293037 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:02.293868 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:02.293876 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:02.294208 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:02.294982 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 01:00:02.295020 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0924 01:00:02.297105 (+ 2085us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 01:00:02.300133 (+ 3028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:00:02.300142 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:00:02.300157 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:00:02.300558 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:00:02.300926 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:00:02.300931 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:00:02.300935 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:00:02.301117 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:00:02.301628 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 01:00:02.301636 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 01:00:02.301957 (+ 321us) client_negotiation.cc:769] Sending connection context
0924 01:00:02.302218 (+ 261us) client_negotiation.cc:240] Negotiation successful
0924 01:00:02.302745 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1025,"thread_start_us":797,"threads_started":1}
I20260924 01:00:02.305370 24870 heartbeater.cc:347] Connected to a master server at 127.8.64.190:42983
I20260924 01:00:02.305629 24870 heartbeater.cc:464] Registering TS with master...
I20260924 01:00:02.306270 24870 heartbeater.cc:511] Master 127.8.64.190:42983 requested a full tablet report, sending...
I20260924 01:00:02.307744 24553 ts_manager.cc:194] Registered new tserver with Master: ae77c9ce9b674ebbb2f57f0bcc293036 (127.8.64.130:43121)
I20260924 01:00:02.308604 24553 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.130@KRBTEST.COM'} at 127.8.64.130:42205
W20260924 01:00:02.617956 24873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:00:02.618242 24873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:00:02.618417 24873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:00:02.625671 24873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:00:02.625777 24873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:00:02.625943 24873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.131
I20260924 01:00:02.635658 24873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:35317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab
--principal=kudu/127.8.64.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.131:38205
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.64.131
--webserver_port=43801
--tserver_master_addrs=127.8.64.190:42983
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.131
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:00:02.637041 24873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:00:02.638432 24873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:00:02.651388 24881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:02.651430 24879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:02.651458 24878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:00:02.652752 24873 server_base.cc:1034] running on GCE node
I20260924 01:00:02.653263 24873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:00:02.654101 24873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:00:02.655287 24873 hybrid_clock.cc:648] HybridClock initialized: now 1790211602655239 us; error 52 us; skew 500 ppm
I20260924 01:00:02.655596 24873 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 01:00:02 dist-test-slave-mmhs krb5kdc[23967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211602, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 01:00:02.658574 24873 init.cc:377] Logged in from keytab as kudu/127.8.64.131@KRBTEST.COM (short username kudu)
I20260924 01:00:02.660070 24873 webserver.cc:460] Webserver started at http://127.8.64.131:43801/ using document root <none> and password file <none>
I20260924 01:00:02.660795 24873 fs_manager.cc:359] Metadata directory not provided
I20260924 01:00:02.660956 24873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:00:02.666144 24873 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260924 01:00:02.668871 24888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:02.670214 24873 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260924 01:00:02.670416 24873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-2/wal
uuid: "4a1609a2bbb742e4b6722861bd63c4a8"
format_stamp: "Formatted at 2026-09-24 00:59:39 on dist-test-slave-mmhs"
server_key: "1af4b25f4ad11c0ca38b5f35e4758e9a"
server_key_iv: "4e2c781d84beede4e8e20109638f010f"
server_key_version: "encryptionkey@0"
I20260924 01:00:02.671443 24873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:00:02.688083 24873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:00:02.692144 24873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:00:02.692418 24873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:00:02.693393 24873 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 01:00:02.694782 24873 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 01:00:02.694900 24873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:02.695034 24873 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 01:00:02.695132 24873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:02.729342 24873 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.131:38205
I20260924 01:00:02.729429 25000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.131:38205 every 8 connection(s)
I20260924 01:00:02.730871 24873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
I20260924 01:00:02.732223 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 24873
Sep 24 01:00:02 dist-test-slave-mmhs krb5kdc[23967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211602, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.131@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 01:00:02.749471 24740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:02.733269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45006 (local address 127.8.64.190:42983)
0924 01:00:02.733460 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:02.733470 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:02.734134 (+ 664us) server_negotiation.cc:407] Connection header received
0924 01:00:02.734380 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:02.734391 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:02.734488 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:02.734650 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:02.736733 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:02.737543 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:02.738995 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:02.739448 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:02.739638 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:02.743333 (+ 3695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:02.743338 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:02.743342 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:00:02.743376 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:02.746258 (+ 2882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:02.747041 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:02.747047 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:02.747052 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:02.747232 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:02.747674 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:02.747680 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:02.747684 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:02.748117 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:02.748435 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:02.748791 (+ 356us) server_negotiation.cc:299] Negotiation successful
0924 01:00:02.749016 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260924 01:00:02.754204 25004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:02.733588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:53193 (local address 127.8.64.190:42983)
0924 01:00:02.736876 (+ 3288us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:02.736882 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:02.736902 (+ 20us) server_negotiation.cc:407] Connection header received
0924 01:00:02.737664 (+ 762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:02.737669 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:02.737745 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:02.737867 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:02.739358 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:02.740177 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:02.742917 (+ 2740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:02.743352 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:02.743495 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:02.747332 (+ 3837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:02.747337 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:02.747340 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:00:02.747371 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:02.750709 (+ 3338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:02.751596 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:02.751602 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:02.751606 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:02.751770 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:02.752294 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:02.752299 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:02.752303 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:02.752591 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:02.752756 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:02.753527 (+ 771us) server_negotiation.cc:299] Negotiation successful
0924 01:00:02.753764 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3153,"thread_start_us":900,"threads_started":1}
I20260924 01:00:02.755311 25005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:02.734146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.131:53193)
0924 01:00:02.735137 (+ 991us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:02.735196 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:02.737404 (+ 2208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:02.738100 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:02.738108 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:02.738647 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0924 01:00:02.739184 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:02.739201 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:02.740514 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:02.740523 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:02.742758 (+ 2235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:02.742775 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:02.743619 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:02.743628 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:02.743858 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:02.744542 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 01:00:02.744567 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0924 01:00:02.747063 (+ 2496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 01:00:02.750924 (+ 3861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:00:02.750947 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:00:02.750967 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:00:02.751434 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:00:02.751888 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:00:02.751894 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:00:02.751898 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:00:02.752176 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:00:02.752764 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 01:00:02.752775 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 01:00:02.753148 (+ 373us) client_negotiation.cc:769] Sending connection context
0924 01:00:02.753422 (+ 274us) client_negotiation.cc:240] Negotiation successful
0924 01:00:02.753999 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"thread_start_us":380,"threads_started":1}
I20260924 01:00:02.756744 25001 heartbeater.cc:347] Connected to a master server at 127.8.64.190:42983
I20260924 01:00:02.757076 25001 heartbeater.cc:464] Registering TS with master...
I20260924 01:00:02.757786 25001 heartbeater.cc:511] Master 127.8.64.190:42983 requested a full tablet report, sending...
I20260924 01:00:02.759302 24553 ts_manager.cc:194] Registered new tserver with Master: 4a1609a2bbb742e4b6722861bd63c4a8 (127.8.64.131:38205)
I20260924 01:00:02.760301 24553 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.131@KRBTEST.COM'} at 127.8.64.131:53193
I20260924 01:00:02.764509 8450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 01:00:02.772100 25004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:02.765479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45014 (local address 127.8.64.190:42983)
0924 01:00:02.765735 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:02.765745 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:02.766201 (+ 456us) server_negotiation.cc:407] Connection header received
0924 01:00:02.766581 (+ 380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:02.766585 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:02.766739 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:02.766899 (+ 160us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 01:00:02.767414 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:02.768215 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:02.769864 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:02.770413 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:02.770556 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:02.771109 (+ 553us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 01:00:02.771322 (+ 213us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0924 01:00:02.771766 (+ 444us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45014: token expired
Metrics: {"server-negotiator.queue_time_us":86}
W20260924 01:00:02.772258 25004 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45014: token expired
W20260924 01:00:02.772769 24472 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.8.64.190:42983)
I20260924 01:00:02.785858 25004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:02.773290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45020 (local address 127.8.64.190:42983)
0924 01:00:02.773472 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:02.773477 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:02.773495 (+ 18us) server_negotiation.cc:407] Connection header received
0924 01:00:02.773709 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:02.773713 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:02.773769 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:02.773865 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:02.775347 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:02.776193 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:02.777534 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:02.777925 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:02.778070 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:02.780693 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:02.780698 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:02.780702 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:00:02.780733 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:02.783229 (+ 2496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:02.783898 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:02.783910 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:02.783913 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:02.784059 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:02.784443 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:02.784449 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:02.784452 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:02.784752 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:02.785020 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:02.785303 (+ 283us) server_negotiation.cc:299] Negotiation successful
0924 01:00:02.785526 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260924 01:00:02.789613 24553 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45020:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 01:00:02.797220 24553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 01:00:02.834815 25013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:02.809515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.130:43121 (local address 127.0.0.1:34534)
0924 01:00:02.822728 (+ 13213us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:02.822747 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:02.822883 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:02.824557 (+ 1674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:02.824568 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:02.824609 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 01:00:02.825612 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:02.825636 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:02.828245 (+ 2609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:02.828254 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:02.831417 (+ 3163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:02.831438 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:02.833640 (+ 2202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:02.833646 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:02.833853 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:02.833889 (+ 36us) client_negotiation.cc:769] Sending connection context
0924 01:00:02.834042 (+ 153us) client_negotiation.cc:240] Negotiation successful
0924 01:00:02.834262 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13080,"mutex_wait_us":278,"spinlock_wait_cycles":1408,"thread_start_us":12913,"threads_started":1}
I20260924 01:00:02.834821 25014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:02.809721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34534 (local address 127.8.64.130:43121)
0924 01:00:02.823444 (+ 13723us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:02.823457 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:02.823490 (+ 33us) server_negotiation.cc:407] Connection header received
0924 01:00:02.823609 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:02.823621 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:02.823946 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:02.824200 (+ 254us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 01:00:02.825900 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:02.828028 (+ 2128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:02.831657 (+ 3629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:02.833410 (+ 1753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:02.833536 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:02.833753 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:02.834285 (+ 532us) server_negotiation.cc:299] Negotiation successful
0924 01:00:02.834470 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13551,"thread_start_us":13451,"threads_started":1}
I20260924 01:00:02.837337 25012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:02.809387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52558 (local address 127.8.64.129:37443)
0924 01:00:02.823860 (+ 14473us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:02.823869 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:02.823895 (+ 26us) server_negotiation.cc:407] Connection header received
0924 01:00:02.824060 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:02.824070 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:02.824307 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:02.824581 (+ 274us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 01:00:02.829574 (+ 4993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:02.832201 (+ 2627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:02.834825 (+ 2624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:02.836246 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:02.836405 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:02.836628 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:02.836849 (+ 221us) server_negotiation.cc:299] Negotiation successful
0924 01:00:02.837030 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14336,"thread_start_us":14246,"threads_started":1}
I20260924 01:00:02.837338 25011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:02.809172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:37443 (local address 127.0.0.1:52558)
0924 01:00:02.822599 (+ 13427us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:02.822664 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:02.822883 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:02.824823 (+ 1940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:02.824828 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:02.824855 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 01:00:02.829130 (+ 4275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:02.829145 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:02.832427 (+ 3282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:02.832431 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:02.834627 (+ 2196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:02.834645 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:02.836463 (+ 1818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:02.836470 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:02.836635 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:02.836678 (+ 43us) client_negotiation.cc:769] Sending connection context
0924 01:00:02.836757 (+ 79us) client_negotiation.cc:240] Negotiation successful
0924 01:00:02.836912 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12366,"mutex_wait_us":3,"thread_start_us":12191,"threads_started":1}
I20260924 01:00:02.839000 24805 tablet_service.cc:1467] Processing CreateTablet for tablet 28c04e23639c4318a322edab1264552d (DEFAULT_TABLE table=test-table [id=8338748c4a234702b19e6ef623a8576f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 01:00:02.840432 24805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28c04e23639c4318a322edab1264552d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:00:02.841147 25015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:02.823250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53086 (local address 127.8.64.131:38205)
0924 01:00:02.832901 (+ 9651us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:02.832912 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:02.833357 (+ 445us) server_negotiation.cc:407] Connection header received
0924 01:00:02.833552 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:02.833559 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:02.833785 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:02.833989 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 01:00:02.834665 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:02.836628 (+ 1963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:02.838888 (+ 2260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:02.839889 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:02.840039 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:02.840248 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:02.840774 (+ 526us) server_negotiation.cc:299] Negotiation successful
0924 01:00:02.840903 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9484,"thread_start_us":9299,"threads_started":1}
I20260924 01:00:02.841639 25016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:02.825297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.131:38205 (local address 127.0.0.1:53086)
0924 01:00:02.833223 (+ 7926us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:02.833243 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:02.833375 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:02.833991 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:02.834008 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:02.834038 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 01:00:02.834440 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:02.834456 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:02.836758 (+ 2302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:02.836764 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:02.838715 (+ 1951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:02.838729 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:02.840092 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:02.840098 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:02.840273 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:02.840312 (+ 39us) client_negotiation.cc:769] Sending connection context
0924 01:00:02.840385 (+ 73us) client_negotiation.cc:240] Negotiation successful
0924 01:00:02.841361 (+ 976us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7785,"thread_start_us":7652,"threads_started":1}
I20260924 01:00:02.841243 24673 tablet_service.cc:1467] Processing CreateTablet for tablet 28c04e23639c4318a322edab1264552d (DEFAULT_TABLE table=test-table [id=8338748c4a234702b19e6ef623a8576f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 01:00:02.842572 24673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28c04e23639c4318a322edab1264552d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:00:02.844429 24936 tablet_service.cc:1467] Processing CreateTablet for tablet 28c04e23639c4318a322edab1264552d (DEFAULT_TABLE table=test-table [id=8338748c4a234702b19e6ef623a8576f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 01:00:02.845626 24936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28c04e23639c4318a322edab1264552d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:00:02.855883 25017 tablet_bootstrap.cc:492] T 28c04e23639c4318a322edab1264552d P ae77c9ce9b674ebbb2f57f0bcc293036: Bootstrap starting.
I20260924 01:00:02.858162 25019 tablet_bootstrap.cc:492] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be: Bootstrap starting.
I20260924 01:00:02.859726 25018 tablet_bootstrap.cc:492] T 28c04e23639c4318a322edab1264552d P 4a1609a2bbb742e4b6722861bd63c4a8: Bootstrap starting.
I20260924 01:00:02.861742 25017 tablet_bootstrap.cc:654] T 28c04e23639c4318a322edab1264552d P ae77c9ce9b674ebbb2f57f0bcc293036: Neither blocks nor log segments found. Creating new log.
I20260924 01:00:02.861769 25019 tablet_bootstrap.cc:654] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be: Neither blocks nor log segments found. Creating new log.
I20260924 01:00:02.862857 25017 log.cc:824] T 28c04e23639c4318a322edab1264552d P ae77c9ce9b674ebbb2f57f0bcc293036: Log is configured to *not* fsync() on all Append() calls
I20260924 01:00:02.862996 25019 log.cc:824] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be: Log is configured to *not* fsync() on all Append() calls
I20260924 01:00:02.863270 25018 tablet_bootstrap.cc:654] T 28c04e23639c4318a322edab1264552d P 4a1609a2bbb742e4b6722861bd63c4a8: Neither blocks nor log segments found. Creating new log.
I20260924 01:00:02.864240 24738 heartbeater.cc:503] Master 127.8.64.190:42983 was elected leader, sending a full tablet report...
I20260924 01:00:02.864519 25018 log.cc:824] T 28c04e23639c4318a322edab1264552d P 4a1609a2bbb742e4b6722861bd63c4a8: Log is configured to *not* fsync() on all Append() calls
I20260924 01:00:02.866195 25017 tablet_bootstrap.cc:492] T 28c04e23639c4318a322edab1264552d P ae77c9ce9b674ebbb2f57f0bcc293036: No bootstrap required, opened a new log
I20260924 01:00:02.866503 25017 ts_tablet_manager.cc:1397] T 28c04e23639c4318a322edab1264552d P ae77c9ce9b674ebbb2f57f0bcc293036: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260924 01:00:02.866951 25019 tablet_bootstrap.cc:492] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be: No bootstrap required, opened a new log
I20260924 01:00:02.867225 25019 ts_tablet_manager.cc:1397] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260924 01:00:02.867705 25018 tablet_bootstrap.cc:492] T 28c04e23639c4318a322edab1264552d P 4a1609a2bbb742e4b6722861bd63c4a8: No bootstrap required, opened a new log
I20260924 01:00:02.867998 25018 ts_tablet_manager.cc:1397] T 28c04e23639c4318a322edab1264552d P 4a1609a2bbb742e4b6722861bd63c4a8: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260924 01:00:02.875309 25019 raft_consensus.cc:348] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4727a59d1809474dad8a93ba1bf545be" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37443 } } peers { permanent_uuid: "ae77c9ce9b674ebbb2f57f0bcc293036" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 43121 } } peers { permanent_uuid: "4a1609a2bbb742e4b6722861bd63c4a8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 38205 } }
I20260924 01:00:02.875761 25019 raft_consensus.cc:374] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 01:00:02.875862 25019 raft_consensus.cc:729] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4727a59d1809474dad8a93ba1bf545be, State: Initialized, Role: FOLLOWER
I20260924 01:00:02.876500 25019 consensus_queue.cc:260] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4727a59d1809474dad8a93ba1bf545be" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37443 } } peers { permanent_uuid: "ae77c9ce9b674ebbb2f57f0bcc293036" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 43121 } } peers { permanent_uuid: "4a1609a2bbb742e4b6722861bd63c4a8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 38205 } }
I20260924 01:00:02.877421 25017 raft_consensus.cc:348] T 28c04e23639c4318a322edab1264552d P ae77c9ce9b674ebbb2f57f0bcc293036 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4727a59d1809474dad8a93ba1bf545be" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37443 } } peers { permanent_uuid: "ae77c9ce9b674ebbb2f57f0bcc293036" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 43121 } } peers { permanent_uuid: "4a1609a2bbb742e4b6722861bd63c4a8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 38205 } }
I20260924 01:00:02.877861 25017 raft_consensus.cc:374] T 28c04e23639c4318a322edab1264552d P ae77c9ce9b674ebbb2f57f0bcc293036 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 01:00:02.877957 25017 raft_consensus.cc:729] T 28c04e23639c4318a322edab1264552d P ae77c9ce9b674ebbb2f57f0bcc293036 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae77c9ce9b674ebbb2f57f0bcc293036, State: Initialized, Role: FOLLOWER
I20260924 01:00:02.878274 25019 ts_tablet_manager.cc:1428] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260924 01:00:02.878682 25017 consensus_queue.cc:260] T 28c04e23639c4318a322edab1264552d P ae77c9ce9b674ebbb2f57f0bcc293036 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4727a59d1809474dad8a93ba1bf545be" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37443 } } peers { permanent_uuid: "ae77c9ce9b674ebbb2f57f0bcc293036" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 43121 } } peers { permanent_uuid: "4a1609a2bbb742e4b6722861bd63c4a8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 38205 } }
I20260924 01:00:02.878643 25018 raft_consensus.cc:348] T 28c04e23639c4318a322edab1264552d P 4a1609a2bbb742e4b6722861bd63c4a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4727a59d1809474dad8a93ba1bf545be" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37443 } } peers { permanent_uuid: "ae77c9ce9b674ebbb2f57f0bcc293036" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 43121 } } peers { permanent_uuid: "4a1609a2bbb742e4b6722861bd63c4a8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 38205 } }
I20260924 01:00:02.879034 25018 raft_consensus.cc:374] T 28c04e23639c4318a322edab1264552d P 4a1609a2bbb742e4b6722861bd63c4a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 01:00:02.879161 25018 raft_consensus.cc:729] T 28c04e23639c4318a322edab1264552d P 4a1609a2bbb742e4b6722861bd63c4a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a1609a2bbb742e4b6722861bd63c4a8, State: Initialized, Role: FOLLOWER
I20260924 01:00:02.879863 25018 consensus_queue.cc:260] T 28c04e23639c4318a322edab1264552d P 4a1609a2bbb742e4b6722861bd63c4a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4727a59d1809474dad8a93ba1bf545be" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37443 } } peers { permanent_uuid: "ae77c9ce9b674ebbb2f57f0bcc293036" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 43121 } } peers { permanent_uuid: "4a1609a2bbb742e4b6722861bd63c4a8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 38205 } }
I20260924 01:00:02.881397 25001 heartbeater.cc:503] Master 127.8.64.190:42983 was elected leader, sending a full tablet report...
I20260924 01:00:02.881899 25018 ts_tablet_manager.cc:1428] T 28c04e23639c4318a322edab1264552d P 4a1609a2bbb742e4b6722861bd63c4a8: Time spent starting tablet: real 0.014s user 0.007s sys 0.005s
I20260924 01:00:02.882915 24870 heartbeater.cc:503] Master 127.8.64.190:42983 was elected leader, sending a full tablet report...
I20260924 01:00:02.883613 25017 ts_tablet_manager.cc:1428] T 28c04e23639c4318a322edab1264552d P ae77c9ce9b674ebbb2f57f0bcc293036: Time spent starting tablet: real 0.017s user 0.004s sys 0.012s
I20260924 01:00:02.920557 25025 raft_consensus.cc:482] T 28c04e23639c4318a322edab1264552d P 4a1609a2bbb742e4b6722861bd63c4a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 01:00:02.920956 25025 raft_consensus.cc:504] T 28c04e23639c4318a322edab1264552d P 4a1609a2bbb742e4b6722861bd63c4a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4727a59d1809474dad8a93ba1bf545be" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37443 } } peers { permanent_uuid: "ae77c9ce9b674ebbb2f57f0bcc293036" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 43121 } } peers { permanent_uuid: "4a1609a2bbb742e4b6722861bd63c4a8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 38205 } }
I20260924 01:00:02.922945 25025 leader_election.cc:288] T 28c04e23639c4318a322edab1264552d P 4a1609a2bbb742e4b6722861bd63c4a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4727a59d1809474dad8a93ba1bf545be (127.8.64.129:37443), ae77c9ce9b674ebbb2f57f0bcc293036 (127.8.64.130:43121)
I20260924 01:00:02.929045 25012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:02.923452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:56929 (local address 127.8.64.129:37443)
0924 01:00:02.923701 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:02.923706 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:02.923725 (+ 19us) server_negotiation.cc:407] Connection header received
0924 01:00:02.923964 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:02.923969 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:02.924033 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:02.924169 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 01:00:02.924685 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:02.925414 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:02.927354 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:02.928145 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:02.928323 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:02.928403 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:02.928664 (+ 261us) server_negotiation.cc:299] Negotiation successful
0924 01:00:02.928798 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260924 01:00:02.929046 25005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:02.923372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:37443 (local address 127.8.64.131:56929)
0924 01:00:02.923618 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:02.923641 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:02.923777 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:02.924182 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:02.924186 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:02.924213 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 01:00:02.924429 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:02.924440 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:02.925560 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:02.925568 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:02.927181 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:02.927193 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:02.928327 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:02.928333 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:02.928504 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:02.928528 (+ 24us) client_negotiation.cc:769] Sending connection context
0924 01:00:02.928637 (+ 109us) client_negotiation.cc:240] Negotiation successful
0924 01:00:02.928770 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260924 01:00:02.930080 25014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:02.923456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:54315 (local address 127.8.64.130:43121)
0924 01:00:02.923644 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:02.923649 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:02.924753 (+ 1104us) server_negotiation.cc:407] Connection header received
0924 01:00:02.924950 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:02.924954 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:02.925016 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:02.925113 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 01:00:02.925629 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:02.926355 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:02.927981 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:02.928947 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:02.929127 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:02.929220 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:02.929695 (+ 475us) server_negotiation.cc:299] Negotiation successful
0924 01:00:02.929830 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260924 01:00:02.930114 25026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:02.923794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.130:43121 (local address 127.8.64.131:54315)
0924 01:00:02.924624 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:02.924642 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:02.924799 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:02.925126 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:02.925130 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:02.925153 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 01:00:02.925464 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:02.925471 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:02.926490 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:02.926497 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:02.927818 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:02.927828 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:02.929420 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:02.929426 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:02.929539 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:02.929561 (+ 22us) client_negotiation.cc:769] Sending connection context
0924 01:00:02.929667 (+ 106us) client_negotiation.cc:240] Negotiation successful
0924 01:00:02.929783 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":374,"threads_started":1}
I20260924 01:00:02.930121 24693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28c04e23639c4318a322edab1264552d" candidate_uuid: "4a1609a2bbb742e4b6722861bd63c4a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4727a59d1809474dad8a93ba1bf545be" is_pre_election: true
I20260924 01:00:02.930778 24693 raft_consensus.cc:2454] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a1609a2bbb742e4b6722861bd63c4a8 in term 0.
I20260924 01:00:02.931186 24825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28c04e23639c4318a322edab1264552d" candidate_uuid: "4a1609a2bbb742e4b6722861bd63c4a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae77c9ce9b674ebbb2f57f0bcc293036" is_pre_election: true
I20260924 01:00:02.931602 24892 leader_election.cc:302] T 28c04e23639c4318a322edab1264552d P 4a1609a2bbb742e4b6722861bd63c4a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4727a59d1809474dad8a93ba1bf545be, 4a1609a2bbb742e4b6722861bd63c4a8; no voters:
I20260924 01:00:02.931741 24825 raft_consensus.cc:2454] T 28c04e23639c4318a322edab1264552d P ae77c9ce9b674ebbb2f57f0bcc293036 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a1609a2bbb742e4b6722861bd63c4a8 in term 0.
I20260924 01:00:02.932065 25025 raft_consensus.cc:2781] T 28c04e23639c4318a322edab1264552d P 4a1609a2bbb742e4b6722861bd63c4a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 01:00:02.932253 25025 raft_consensus.cc:482] T 28c04e23639c4318a322edab1264552d P 4a1609a2bbb742e4b6722861bd63c4a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 01:00:02.932368 25025 raft_consensus.cc:3037] T 28c04e23639c4318a322edab1264552d P 4a1609a2bbb742e4b6722861bd63c4a8 [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:00:02.934666 25025 raft_consensus.cc:504] T 28c04e23639c4318a322edab1264552d P 4a1609a2bbb742e4b6722861bd63c4a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4727a59d1809474dad8a93ba1bf545be" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37443 } } peers { permanent_uuid: "ae77c9ce9b674ebbb2f57f0bcc293036" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 43121 } } peers { permanent_uuid: "4a1609a2bbb742e4b6722861bd63c4a8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 38205 } }
I20260924 01:00:02.935241 25025 leader_election.cc:288] T 28c04e23639c4318a322edab1264552d P 4a1609a2bbb742e4b6722861bd63c4a8 [CANDIDATE]: Term 1 election: Requested vote from peers 4727a59d1809474dad8a93ba1bf545be (127.8.64.129:37443), ae77c9ce9b674ebbb2f57f0bcc293036 (127.8.64.130:43121)
I20260924 01:00:02.935674 24693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28c04e23639c4318a322edab1264552d" candidate_uuid: "4a1609a2bbb742e4b6722861bd63c4a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4727a59d1809474dad8a93ba1bf545be"
I20260924 01:00:02.935726 24825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28c04e23639c4318a322edab1264552d" candidate_uuid: "4a1609a2bbb742e4b6722861bd63c4a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae77c9ce9b674ebbb2f57f0bcc293036"
I20260924 01:00:02.935897 24693 raft_consensus.cc:3037] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:00:02.935963 24825 raft_consensus.cc:3037] T 28c04e23639c4318a322edab1264552d P ae77c9ce9b674ebbb2f57f0bcc293036 [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:00:02.938184 24693 raft_consensus.cc:2454] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a1609a2bbb742e4b6722861bd63c4a8 in term 1.
I20260924 01:00:02.938191 24825 raft_consensus.cc:2454] T 28c04e23639c4318a322edab1264552d P ae77c9ce9b674ebbb2f57f0bcc293036 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a1609a2bbb742e4b6722861bd63c4a8 in term 1.
I20260924 01:00:02.938627 24892 leader_election.cc:302] T 28c04e23639c4318a322edab1264552d P 4a1609a2bbb742e4b6722861bd63c4a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4727a59d1809474dad8a93ba1bf545be, 4a1609a2bbb742e4b6722861bd63c4a8; no voters:
I20260924 01:00:02.938942 25025 raft_consensus.cc:2781] T 28c04e23639c4318a322edab1264552d P 4a1609a2bbb742e4b6722861bd63c4a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 01:00:02.939407 25025 raft_consensus.cc:686] T 28c04e23639c4318a322edab1264552d P 4a1609a2bbb742e4b6722861bd63c4a8 [term 1 LEADER]: Becoming Leader. State: Replica: 4a1609a2bbb742e4b6722861bd63c4a8, State: Running, Role: LEADER
I20260924 01:00:02.939877 25025 consensus_queue.cc:237] T 28c04e23639c4318a322edab1264552d P 4a1609a2bbb742e4b6722861bd63c4a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4727a59d1809474dad8a93ba1bf545be" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37443 } } peers { permanent_uuid: "ae77c9ce9b674ebbb2f57f0bcc293036" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 43121 } } peers { permanent_uuid: "4a1609a2bbb742e4b6722861bd63c4a8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 38205 } }
I20260924 01:00:02.944304 24553 catalog_manager.cc:5495] T 28c04e23639c4318a322edab1264552d P 4a1609a2bbb742e4b6722861bd63c4a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4a1609a2bbb742e4b6722861bd63c4a8 (127.8.64.131). New cstate: current_term: 1 leader_uuid: "4a1609a2bbb742e4b6722861bd63c4a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4727a59d1809474dad8a93ba1bf545be" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae77c9ce9b674ebbb2f57f0bcc293036" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 43121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a1609a2bbb742e4b6722861bd63c4a8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 38205 } health_report { overall_health: HEALTHY } } }
I20260924 01:00:02.974524 25015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:02.969455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53096 (local address 127.8.64.131:38205)
0924 01:00:02.969648 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:02.969652 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:02.969668 (+ 16us) server_negotiation.cc:407] Connection header received
0924 01:00:02.969753 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:02.969756 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:02.969829 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:02.969952 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 01:00:02.970366 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:02.971104 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:02.972514 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:02.972942 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:02.973039 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:02.973516 (+ 477us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 01:00:02.973993 (+ 477us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 01:00:02.974112 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:02.974201 (+ 89us) server_negotiation.cc:299] Negotiation successful
0924 01:00:02.974264 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260924 01:00:02.982390 25002 tablet.cc:2369] T 28c04e23639c4318a322edab1264552d P 4a1609a2bbb742e4b6722861bd63c4a8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 01:00:02.982587 24825 raft_consensus.cc:1261] T 28c04e23639c4318a322edab1264552d P ae77c9ce9b674ebbb2f57f0bcc293036 [term 1 FOLLOWER]: Refusing update from remote peer 4a1609a2bbb742e4b6722861bd63c4a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 01:00:02.982589 24693 raft_consensus.cc:1261] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [term 1 FOLLOWER]: Refusing update from remote peer 4a1609a2bbb742e4b6722861bd63c4a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 01:00:02.983624 25027 consensus_queue.cc:1035] T 28c04e23639c4318a322edab1264552d P 4a1609a2bbb742e4b6722861bd63c4a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae77c9ce9b674ebbb2f57f0bcc293036" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 43121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 01:00:02.983949 25025 consensus_queue.cc:1035] T 28c04e23639c4318a322edab1264552d P 4a1609a2bbb742e4b6722861bd63c4a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4727a59d1809474dad8a93ba1bf545be" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 01:00:02.995160 25030 mvcc.cc:205] Tried to move back new op lower bound from 7332706725806522368 to 7332706725645565952. Current Snapshot: MvccSnapshot[applied={T|T < 7332706725806522368}]
I20260924 01:00:03.009450 24552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 01:00:03.009778 24552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 01:00:03.013799 24552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770: Sending DeleteTablet for 3 replicas of tablet 28c04e23639c4318a322edab1264552d
I20260924 01:00:03.015004 24673 tablet_service.cc:1514] Processing DeleteTablet for tablet 28c04e23639c4318a322edab1264552d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 01:00:03 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:52558
I20260924 01:00:03.015385 24936 tablet_service.cc:1514] Processing DeleteTablet for tablet 28c04e23639c4318a322edab1264552d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 01:00:03 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:53086
I20260924 01:00:03.015766 24805 tablet_service.cc:1514] Processing DeleteTablet for tablet 28c04e23639c4318a322edab1264552d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 01:00:03 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:34534
I20260924 01:00:03.016647 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 24590
I20260924 01:00:03.040182 25041 tablet_replica.cc:330] T 28c04e23639c4318a322edab1264552d P ae77c9ce9b674ebbb2f57f0bcc293036: stopping tablet replica
I20260924 01:00:03.040675 25041 raft_consensus.cc:2229] T 28c04e23639c4318a322edab1264552d P ae77c9ce9b674ebbb2f57f0bcc293036 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 01:00:03.041395 25041 raft_consensus.cc:2258] T 28c04e23639c4318a322edab1264552d P ae77c9ce9b674ebbb2f57f0bcc293036 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 01:00:03.042336 25039 tablet_replica.cc:330] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be: stopping tablet replica
I20260924 01:00:03.042769 25039 raft_consensus.cc:2229] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 01:00:03.043298 25039 raft_consensus.cc:2258] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 01:00:03.044636 25012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:03.017874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52566 (local address 127.8.64.129:37443)
0924 01:00:03.018661 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:03.018670 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:03.018692 (+ 22us) server_negotiation.cc:407] Connection header received
0924 01:00:03.018769 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:03.018777 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:03.018911 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:03.019026 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:03.021643 (+ 2617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:03.023297 (+ 1654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:03.025998 (+ 2701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:03.026962 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:03.027132 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:03.032146 (+ 5014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:03.032174 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:03.032203 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:00:03.032268 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:03.037883 (+ 5615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:03.039292 (+ 1409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:03.039306 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:03.039312 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:03.039534 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:03.040654 (+ 1120us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:03.040663 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:03.040669 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:03.041647 (+ 978us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:03.042356 (+ 709us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:03.043698 (+ 1342us) server_negotiation.cc:299] Negotiation successful
0924 01:00:03.044042 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623}
I20260924 01:00:03.045662 25039 ts_tablet_manager.cc:1905] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 01:00:03.046999 24733 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 01:00:03.047662 25040 tablet_replica.cc:330] T 28c04e23639c4318a322edab1264552d P 4a1609a2bbb742e4b6722861bd63c4a8: stopping tablet replica
I20260924 01:00:03.048108 25040 raft_consensus.cc:2229] T 28c04e23639c4318a322edab1264552d P 4a1609a2bbb742e4b6722861bd63c4a8 [term 1 LEADER]: Raft consensus shutting down.
I20260924 01:00:03.048789 25040 raft_consensus.cc:2258] T 28c04e23639c4318a322edab1264552d P 4a1609a2bbb742e4b6722861bd63c4a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 01:00:03.050956 25040 ts_tablet_manager.cc:1905] T 28c04e23639c4318a322edab1264552d P 4a1609a2bbb742e4b6722861bd63c4a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 01:00:03.057374 25041 ts_tablet_manager.cc:1905] T 28c04e23639c4318a322edab1264552d P ae77c9ce9b674ebbb2f57f0bcc293036: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 01:00:03.059597 25040 ts_tablet_manager.cc:1918] T 28c04e23639c4318a322edab1264552d P 4a1609a2bbb742e4b6722861bd63c4a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 01:00:03.059787 25040 log.cc:1196] T 28c04e23639c4318a322edab1264552d P 4a1609a2bbb742e4b6722861bd63c4a8: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-2/wal/wals/28c04e23639c4318a322edab1264552d
I20260924 01:00:03.060376 25040 ts_tablet_manager.cc:1939] T 28c04e23639c4318a322edab1264552d P 4a1609a2bbb742e4b6722861bd63c4a8: Deleting consensus metadata
I20260924 01:00:03.062350 24538 catalog_manager.cc:4841] TS 4a1609a2bbb742e4b6722861bd63c4a8 (127.8.64.131:38205): tablet 28c04e23639c4318a322edab1264552d (table test-table [id=8338748c4a234702b19e6ef623a8576f]) successfully deleted
I20260924 01:00:03.064142 25041 ts_tablet_manager.cc:1918] T 28c04e23639c4318a322edab1264552d P ae77c9ce9b674ebbb2f57f0bcc293036: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 01:00:03.064306 25041 log.cc:1196] T 28c04e23639c4318a322edab1264552d P ae77c9ce9b674ebbb2f57f0bcc293036: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-1/wal/wals/28c04e23639c4318a322edab1264552d
I20260924 01:00:03.064868 25041 ts_tablet_manager.cc:1939] T 28c04e23639c4318a322edab1264552d P ae77c9ce9b674ebbb2f57f0bcc293036: Deleting consensus metadata
I20260924 01:00:03.066882 24538 catalog_manager.cc:4841] TS ae77c9ce9b674ebbb2f57f0bcc293036 (127.8.64.130:43121): tablet 28c04e23639c4318a322edab1264552d (table test-table [id=8338748c4a234702b19e6ef623a8576f]) successfully deleted
I20260924 01:00:03.153316 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 24590
W20260924 01:00:03.171940 24540 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.64.129:37443 (error 108)
I20260924 01:00:03.172703 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 24742
I20260924 01:00:03.174546 25016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:03.173729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:37443 (local address 127.0.0.1:52568)
0924 01:00:03.174146 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:03.174257 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.129:37443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":270}
W20260924 01:00:03.174795 24540 catalog_manager.cc:4584] TS 4727a59d1809474dad8a93ba1bf545be (127.8.64.129:37443): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 28c04e23639c4318a322edab1264552d: Network error: Client connection negotiation failed: client connection to 127.8.64.129:37443: connect: Connection refused (error 111)
I20260924 01:00:03.190704 25014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:03.173676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34550 (local address 127.8.64.130:43121)
0924 01:00:03.174074 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:03.174081 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:03.174103 (+ 22us) server_negotiation.cc:407] Connection header received
0924 01:00:03.174174 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:03.174180 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:03.174267 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:03.174410 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:03.176513 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:03.177253 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:03.179295 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:03.179733 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:03.179839 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:03.183556 (+ 3717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:03.183569 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:03.183605 (+ 36us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:00:03.183650 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:03.186752 (+ 3102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:03.187697 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:03.187713 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:03.187715 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:03.187840 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:03.188348 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:03.188353 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:03.188356 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:03.188916 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:03.189272 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:03.189799 (+ 527us) server_negotiation.cc:299] Negotiation successful
0924 01:00:03.190174 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220}
I20260924 01:00:03.194485 24865 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 01:00:03.228555 25016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:03.227987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:37443 (local address 127.0.0.1:52574)
0924 01:00:03.228241 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:03.228361 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.129:37443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260924 01:00:03.229795 25016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:03.229337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:37443 (local address 127.0.0.1:52588)
0924 01:00:03.229525 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:03.229620 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.129:37443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260924 01:00:03.286613 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 24742
I20260924 01:00:03.290696 25016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:03.290078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:37443 (local address 127.0.0.1:52594)
0924 01:00:03.290379 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:03.290495 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.129:37443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260924 01:00:03.291996 25016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:03.291496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:37443 (local address 127.0.0.1:52596)
0924 01:00:03.291735 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:03.291828 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.129:37443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260924 01:00:03.305357 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 24873
I20260924 01:00:03.324754 25015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:03.310215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53104 (local address 127.8.64.131:38205)
0924 01:00:03.310552 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:03.310557 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:03.310817 (+ 260us) server_negotiation.cc:407] Connection header received
0924 01:00:03.311044 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:03.311049 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:03.311117 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:03.311240 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:03.312907 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:03.313716 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:03.315212 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:03.315711 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:03.315899 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:03.318618 (+ 2719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:03.318630 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:03.318645 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:00:03.318686 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:03.321686 (+ 3000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:03.322444 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:03.322454 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:03.322458 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:03.322643 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:03.323035 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:03.323040 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:03.323043 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:03.323399 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:03.323723 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:03.324048 (+ 325us) server_negotiation.cc:299] Negotiation successful
0924 01:00:03.324269 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215}
I20260924 01:00:03.325480 24996 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 01:00:03.372794 25016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:03.372260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:37443 (local address 127.0.0.1:52600)
0924 01:00:03.372486 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:03.372604 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.129:37443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260924 01:00:03.373953 25016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:03.373575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:37443 (local address 127.0.0.1:52602)
0924 01:00:03.373769 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:03.373841 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.129:37443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260924 01:00:03.447700 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 24873
I20260924 01:00:03.468030 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 24520
I20260924 01:00:03.469233 24583 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 01:00:03.584111 25016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:03.583181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:37443 (local address 127.0.0.1:52618)
0924 01:00:03.583603 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:03.583736 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.129:37443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225}
I20260924 01:00:03.584112 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 24520
I20260924 01:00:23.607669 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:42983
--webserver_interface=127.8.64.190
--webserver_port=34309
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:35317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:42983
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 01:00:23.936529 25047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:00:23.936874 25047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:00:23.937047 25047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:00:23.944283 25047 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260924 01:00:23.944422 25047 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260924 01:00:23.944519 25047 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260924 01:00:23.944631 25047 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 01:00:23.944737 25047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:00:23.944840 25047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:00:23.944916 25047 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 01:00:23.944996 25047 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 01:00:23.954880 25047 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:35317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:42983
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:42983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=34309
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:00:23.956221 25047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:00:23.957567 25047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:00:23.970474 25052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:23.970397 25055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:23.970363 25053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:00:23.970929 25047 server_base.cc:1034] running on GCE node
I20260924 01:00:23.971760 25047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:00:23.973135 25047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:00:23.974481 25047 hybrid_clock.cc:648] HybridClock initialized: now 1790211623974414 us; error 56 us; skew 500 ppm
I20260924 01:00:23.974834 25047 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 01:00:23 dist-test-slave-mmhs krb5kdc[23967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211623, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 01:00:23.985651 25047 init.cc:377] Logged in from keytab as kudu/127.8.64.190@KRBTEST.COM (short username kudu)
I20260924 01:00:23.987335 25047 webserver.cc:460] Webserver started at http://127.8.64.190:34309/ using document root <none> and password file <none>
I20260924 01:00:23.988075 25047 fs_manager.cc:359] Metadata directory not provided
I20260924 01:00:23.988171 25047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:00:23.994067 25047 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260924 01:00:23.997059 25062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:23.998701 25047 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 01:00:23.998852 25047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "1ffc1d56a39440f09aa42a5e0174c770"
format_stamp: "Formatted at 2026-09-24 00:59:38 on dist-test-slave-mmhs"
server_key: "04e8f38f2e0aaffd2381c2ef1e98b816"
server_key_iv: "6ecfc60af2ee8cc0642ed76bbe6f88ce"
server_key_version: "encryptionkey@0"
I20260924 01:00:23.999861 25047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:00:24.031852 25047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:00:24.036798 25047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:00:24.037065 25047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:00:24.058571 25047 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.190:42983
I20260924 01:00:24.058614 25113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.190:42983 every 8 connection(s)
I20260924 01:00:24.060232 25047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 01:00:24.066104 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 25047
I20260924 01:00:24.067162 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.129:37443
--local_ip_for_outbound_sockets=127.8.64.129
--tserver_master_addrs=127.8.64.190:42983
--webserver_port=36865
--webserver_interface=127.8.64.129
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab
--principal=kudu/127.8.64.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:35317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 01:00:24.067808 25114 sys_catalog.cc:263] Verifying existing consensus state
I20260924 01:00:24.073239 25114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770: Bootstrap starting.
I20260924 01:00:24.094981 25114 log.cc:824] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770: Log is configured to *not* fsync() on all Append() calls
I20260924 01:00:24.112385 25114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 01:00:24.112942 25114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770: Bootstrap complete.
I20260924 01:00:24.122219 25114 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ffc1d56a39440f09aa42a5e0174c770" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42983 } }
I20260924 01:00:24.123210 25114 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ffc1d56a39440f09aa42a5e0174c770, State: Initialized, Role: FOLLOWER
I20260924 01:00:24.123800 25114 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ffc1d56a39440f09aa42a5e0174c770" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42983 } }
I20260924 01:00:24.124055 25114 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 01:00:24.124189 25114 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 01:00:24.124366 25114 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [term 2 FOLLOWER]: Advancing to term 3
I20260924 01:00:24.128384 25114 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ffc1d56a39440f09aa42a5e0174c770" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42983 } }
I20260924 01:00:24.128854 25114 leader_election.cc:302] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ffc1d56a39440f09aa42a5e0174c770; no voters:
I20260924 01:00:24.129539 25114 leader_election.cc:288] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260924 01:00:24.129665 25118 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 01:00:24.130961 25118 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [term 3 LEADER]: Becoming Leader. State: Replica: 1ffc1d56a39440f09aa42a5e0174c770, State: Running, Role: LEADER
I20260924 01:00:24.131490 25118 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ffc1d56a39440f09aa42a5e0174c770" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42983 } }
I20260924 01:00:24.131870 25114 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 01:00:24.133983 25119 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "1ffc1d56a39440f09aa42a5e0174c770" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ffc1d56a39440f09aa42a5e0174c770" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42983 } } }
I20260924 01:00:24.134407 25119 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [sys.catalog]: This master's current role is: LEADER
I20260924 01:00:24.138247 25120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ffc1d56a39440f09aa42a5e0174c770. Latest consensus state: current_term: 3 leader_uuid: "1ffc1d56a39440f09aa42a5e0174c770" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ffc1d56a39440f09aa42a5e0174c770" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42983 } } }
I20260924 01:00:24.138640 25120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770 [sys.catalog]: This master's current role is: LEADER
I20260924 01:00:24.140291 25124 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 01:00:24.149149 25124 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 01:00:24.150714 25124 catalog_manager.cc:1259] Loaded cluster ID: 6318508f6afc4040b7443bf314edebcd
I20260924 01:00:24.150923 25124 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 01:00:24.155385 25124 catalog_manager.cc:1504] Loading token signing keys...
I20260924 01:00:24.157573 25124 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770: Loaded TSK: 0
I20260924 01:00:24.157823 25124 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1ffc1d56a39440f09aa42a5e0174c770: Loaded TSK: 1
F20260924 01:00:24.159107 25124 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fca3afe39ad google::LogMessage::Fail() at ??:0
@ 0x7fca3afe688a google::LogMessage::SendToLog() at ??:0
@ 0x7fca3afe34de google::LogMessage::Flush() at ??:0
@ 0x7fca3afe4a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fca3ed477d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fca3ed3673e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fca50a121c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fca50a11783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fca50a9d2a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7fca50a9d0c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7fca50ae75e9 std::function<>::operator()() at ??:0
@ 0x7fca50a15cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fca50a13c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fca50a9b610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fca50a9b439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fca3dc2f345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fca3dc3291c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fca3dc32799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fca3dc0deaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fca40ee76db start_thread at ??:0
@ 0x7fca39e7d61f clone at ??:0
W20260924 01:00:24.465456 25116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:00:24.465752 25116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:00:24.465938 25116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:00:24.473927 25116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:00:24.474035 25116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:00:24.474261 25116 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.129
I20260924 01:00:24.484288 25116 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:35317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab
--principal=kudu/127.8.64.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.129:37443
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.64.129
--webserver_port=36865
--tserver_master_addrs=127.8.64.190:42983
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.129
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:00:24.485642 25116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:00:24.487097 25116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:00:24.500837 25164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:24.500857 25161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:24.500854 25162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:00:24.506404 25116 server_base.cc:1034] running on GCE node
I20260924 01:00:24.507081 25116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:00:24.508049 25116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:00:24.509297 25116 hybrid_clock.cc:648] HybridClock initialized: now 1790211624509223 us; error 55 us; skew 500 ppm
I20260924 01:00:24.509621 25116 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 01:00:24 dist-test-slave-mmhs krb5kdc[23967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211624, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 01:00:24.512957 25116 init.cc:377] Logged in from keytab as kudu/127.8.64.129@KRBTEST.COM (short username kudu)
I20260924 01:00:24.514676 25116 webserver.cc:460] Webserver started at http://127.8.64.129:36865/ using document root <none> and password file <none>
I20260924 01:00:24.515453 25116 fs_manager.cc:359] Metadata directory not provided
I20260924 01:00:24.515619 25116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:00:24.521288 25116 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260924 01:00:24.524611 25171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:24.526001 25116 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260924 01:00:24.526185 25116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-0/wal
uuid: "4727a59d1809474dad8a93ba1bf545be"
format_stamp: "Formatted at 2026-09-24 00:59:38 on dist-test-slave-mmhs"
server_key: "95872cadad7a1c8ac90e941ca2919790"
server_key_iv: "40db947aad7907bb11b9f1d79ce224b1"
server_key_version: "encryptionkey@0"
I20260924 01:00:24.527215 25116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:00:24.546041 25116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:00:24.550020 25116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:00:24.550280 25116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:00:24.551268 25116 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 01:00:24.553098 25178 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260924 01:00:24.559018 25116 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260924 01:00:24.559141 25116 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.001s
I20260924 01:00:24.559351 25116 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260924 01:00:24.562850 25116 ts_tablet_manager.cc:610] Registered 1 tablets
I20260924 01:00:24.562947 25116 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s user 0.003s sys 0.000s
I20260924 01:00:24.563331 25178 tablet_bootstrap.cc:492] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be: Bootstrap starting.
I20260924 01:00:24.571835 25178 log.cc:824] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be: Log is configured to *not* fsync() on all Append() calls
I20260924 01:00:24.586654 25178 tablet_bootstrap.cc:492] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 01:00:24.587507 25178 tablet_bootstrap.cc:492] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be: Bootstrap complete.
I20260924 01:00:24.587981 25178 ts_tablet_manager.cc:1397] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be: Time spent bootstrapping tablet: real 0.025s user 0.019s sys 0.004s
I20260924 01:00:24.597759 25178 raft_consensus.cc:348] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4727a59d1809474dad8a93ba1bf545be" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37443 } } peers { permanent_uuid: "ae77c9ce9b674ebbb2f57f0bcc293036" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 43121 } } peers { permanent_uuid: "4a1609a2bbb742e4b6722861bd63c4a8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 38205 } }
I20260924 01:00:24.599572 25178 raft_consensus.cc:729] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4727a59d1809474dad8a93ba1bf545be, State: Initialized, Role: FOLLOWER
I20260924 01:00:24.600374 25178 consensus_queue.cc:260] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4727a59d1809474dad8a93ba1bf545be" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37443 } } peers { permanent_uuid: "ae77c9ce9b674ebbb2f57f0bcc293036" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 43121 } } peers { permanent_uuid: "4a1609a2bbb742e4b6722861bd63c4a8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 38205 } }
I20260924 01:00:24.602542 25178 ts_tablet_manager.cc:1428] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be: Time spent starting tablet: real 0.014s user 0.014s sys 0.002s
I20260924 01:00:24.609488 25116 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.129:37443
I20260924 01:00:24.609529 25286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.129:37443 every 8 connection(s)
I20260924 01:00:24.610913 25116 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
I20260924 01:00:24.615875 25289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:24.613836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.129:48197)
0924 01:00:24.614798 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:24.615012 (+ 214us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":718,"thread_start_us":490,"threads_started":1}
W20260924 01:00:24.616623 25287 heartbeater.cc:644] Failed to heartbeat to 127.8.64.190:42983 (0 consecutive failures): Network error: Failed to ping master at 127.8.64.190:42983: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
I20260924 01:00:24.616912 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 25116
I20260924 01:00:24.617507 25289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:24.617182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.129:46459)
0924 01:00:24.617331 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:24.617404 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260924 01:00:24.617642 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.130:43121
--local_ip_for_outbound_sockets=127.8.64.130
--tserver_master_addrs=127.8.64.190:42983
--webserver_port=33007
--webserver_interface=127.8.64.130
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab
--principal=kudu/127.8.64.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:35317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 01:00:24.618546 25289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:24.618008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.129:52801)
0924 01:00:24.618210 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:24.618349 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260924 01:00:24.618841 25287 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 01:00:24.951577 25290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:00:24.951864 25290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:00:24.951958 25290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:00:24.959805 25290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:00:24.959925 25290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:00:24.960042 25290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.130
I20260924 01:00:24.969774 25290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:35317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab
--principal=kudu/127.8.64.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.130:43121
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.64.130
--webserver_port=33007
--tserver_master_addrs=127.8.64.190:42983
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.130
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:00:24.971017 25290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:00:24.972271 25290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:00:24.985105 25296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:24.985224 25295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:24.985704 25298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:00:24.985939 25290 server_base.cc:1034] running on GCE node
I20260924 01:00:24.986661 25290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:00:24.987552 25290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:00:24.988830 25290 hybrid_clock.cc:648] HybridClock initialized: now 1790211624988780 us; error 59 us; skew 500 ppm
I20260924 01:00:24.989188 25290 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 01:00:24 dist-test-slave-mmhs krb5kdc[23967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211624, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 01:00:24.992472 25290 init.cc:377] Logged in from keytab as kudu/127.8.64.130@KRBTEST.COM (short username kudu)
I20260924 01:00:24.994055 25290 webserver.cc:460] Webserver started at http://127.8.64.130:33007/ using document root <none> and password file <none>
I20260924 01:00:24.994843 25290 fs_manager.cc:359] Metadata directory not provided
I20260924 01:00:24.994997 25290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:00:25.000334 25290 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260924 01:00:25.003376 25305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:25.004839 25290 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 01:00:25.005041 25290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-1/wal
uuid: "ae77c9ce9b674ebbb2f57f0bcc293036"
format_stamp: "Formatted at 2026-09-24 00:59:39 on dist-test-slave-mmhs"
server_key: "7ee3772027126ed8f50e0a5594458d7b"
server_key_iv: "867be1ef8dde9ee0daf38d095f63ce6c"
server_key_version: "encryptionkey@0"
I20260924 01:00:25.006034 25290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:00:25.021975 25290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:00:25.026697 25290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:00:25.026991 25290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:00:25.027980 25290 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 01:00:25.029321 25290 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 01:00:25.029464 25290 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:25.029608 25290 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 01:00:25.029700 25290 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:25.064352 25290 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.130:43121
I20260924 01:00:25.064450 25417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.130:43121 every 8 connection(s)
I20260924 01:00:25.065918 25290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
I20260924 01:00:25.070971 25420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:25.068982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.130:38193)
0924 01:00:25.069767 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:25.069977 (+ 210us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":319,"threads_started":1}
W20260924 01:00:25.071661 25418 heartbeater.cc:644] Failed to heartbeat to 127.8.64.190:42983 (0 consecutive failures): Network error: Failed to ping master at 127.8.64.190:42983: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
I20260924 01:00:25.072600 25420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:25.072184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.130:39841)
0924 01:00:25.072372 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:25.072482 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260924 01:00:25.073594 25420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:25.073268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.130:54001)
0924 01:00:25.073404 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:25.073480 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260924 01:00:25.073887 25418 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260924 01:00:25.074195 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 25290
I20260924 01:00:25.074846 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.131:38205
--local_ip_for_outbound_sockets=127.8.64.131
--tserver_master_addrs=127.8.64.190:42983
--webserver_port=43801
--webserver_interface=127.8.64.131
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab
--principal=kudu/127.8.64.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:35317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 01:00:25.406418 25421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:00:25.406711 25421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:00:25.406863 25421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:00:25.414758 25421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:00:25.414878 25421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:00:25.415092 25421 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.131
I20260924 01:00:25.424925 25421 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:35317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab
--principal=kudu/127.8.64.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.131:38205
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.64.131
--webserver_port=43801
--tserver_master_addrs=127.8.64.190:42983
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.131
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:00:25.426260 25421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:00:25.427659 25421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:00:25.441078 25426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:25.441134 25429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:25.441159 25427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:00:25.442150 25421 server_base.cc:1034] running on GCE node
I20260924 01:00:25.442837 25421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:00:25.443787 25421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:00:25.445031 25421 hybrid_clock.cc:648] HybridClock initialized: now 1790211625444995 us; error 52 us; skew 500 ppm
I20260924 01:00:25.445353 25421 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 01:00:25 dist-test-slave-mmhs krb5kdc[23967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211625, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 01:00:25.448534 25421 init.cc:377] Logged in from keytab as kudu/127.8.64.131@KRBTEST.COM (short username kudu)
I20260924 01:00:25.449975 25421 webserver.cc:460] Webserver started at http://127.8.64.131:43801/ using document root <none> and password file <none>
I20260924 01:00:25.450747 25421 fs_manager.cc:359] Metadata directory not provided
I20260924 01:00:25.450897 25421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:00:25.456275 25421 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260924 01:00:25.459075 25436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:25.460386 25421 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 01:00:25.460584 25421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-2/wal
uuid: "4a1609a2bbb742e4b6722861bd63c4a8"
format_stamp: "Formatted at 2026-09-24 00:59:39 on dist-test-slave-mmhs"
server_key: "1af4b25f4ad11c0ca38b5f35e4758e9a"
server_key_iv: "4e2c781d84beede4e8e20109638f010f"
server_key_version: "encryptionkey@0"
I20260924 01:00:25.461545 25421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:00:25.483898 25421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:00:25.488234 25421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:00:25.488550 25421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:00:25.489627 25421 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 01:00:25.491062 25421 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 01:00:25.491192 25421 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:25.491339 25421 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 01:00:25.491432 25421 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:25.528714 25421 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.131:38205
I20260924 01:00:25.528812 25548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.131:38205 every 8 connection(s)
I20260924 01:00:25.530217 25421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
I20260924 01:00:25.532873 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 25421
I20260924 01:00:25.535800 25551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:25.533554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.131:50169)
0924 01:00:25.534588 (+ 1034us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:25.534918 (+ 330us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":671,"thread_start_us":413,"threads_started":1}
W20260924 01:00:25.536541 25549 heartbeater.cc:644] Failed to heartbeat to 127.8.64.190:42983 (0 consecutive failures): Network error: Failed to ping master at 127.8.64.190:42983: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
I20260924 01:00:25.537420 25551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:25.537110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.131:37069)
0924 01:00:25.537253 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:25.537318 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260924 01:00:25.538359 25551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:25.537987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.131:33271)
0924 01:00:25.538124 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:25.538195 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260924 01:00:25.538630 25549 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260924 01:00:25.620273 25553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:25.619511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.129:46029)
0924 01:00:25.619990 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:25.620071 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":229,"threads_started":1}
I20260924 01:00:26.075369 25554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:26.074492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.130:48273)
0924 01:00:26.075042 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:26.075144 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":322,"threads_started":1}
I20260924 01:00:26.414645 25555 raft_consensus.cc:482] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 01:00:26.414996 25555 raft_consensus.cc:504] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4727a59d1809474dad8a93ba1bf545be" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37443 } } peers { permanent_uuid: "ae77c9ce9b674ebbb2f57f0bcc293036" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 43121 } } peers { permanent_uuid: "4a1609a2bbb742e4b6722861bd63c4a8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 38205 } }
I20260924 01:00:26.416682 25555 leader_election.cc:288] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ae77c9ce9b674ebbb2f57f0bcc293036 (127.8.64.130:43121), 4a1609a2bbb742e4b6722861bd63c4a8 (127.8.64.131:38205)
Sep 24 01:00:26 dist-test-slave-mmhs krb5kdc[23967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211624, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for kudu/127.8.64.130@KRBTEST.COM
Sep 24 01:00:26 dist-test-slave-mmhs krb5kdc[23967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211624, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for kudu/127.8.64.131@KRBTEST.COM
I20260924 01:00:26.439615 25556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:26.416953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.130:43121 (local address 127.8.64.129:57241)
0924 01:00:26.417525 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:26.417556 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:26.418236 (+ 680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:26.421929 (+ 3693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:26.421941 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:26.422843 (+ 902us) client_negotiation.cc:189] Negotiated authn=SASL
0924 01:00:26.423550 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:26.423571 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:26.425117 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:26.425121 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:26.426698 (+ 1577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:26.426711 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:26.427707 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:26.427718 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:26.428007 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:26.428846 (+ 839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 01:00:26.428863 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0924 01:00:26.431493 (+ 2630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 01:00:26.435443 (+ 3950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:00:26.435452 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:00:26.435470 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:00:26.436083 (+ 613us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:00:26.436589 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:00:26.436594 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:00:26.436597 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:00:26.436862 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:00:26.437832 (+ 970us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 01:00:26.437840 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 01:00:26.438202 (+ 362us) client_negotiation.cc:769] Sending connection context
0924 01:00:26.438552 (+ 350us) client_negotiation.cc:240] Negotiation successful
0924 01:00:26.439109 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"mutex_wait_us":159,"spinlock_wait_cycles":50048,"thread_start_us":305,"threads_started":1}
I20260924 01:00:26.439688 25559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:26.418270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:56149 (local address 127.8.64.131:38205)
0924 01:00:26.419016 (+ 746us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:26.419034 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:26.419073 (+ 39us) server_negotiation.cc:407] Connection header received
0924 01:00:26.421177 (+ 2104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:26.421189 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:26.421579 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:26.421869 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:26.423763 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:26.424897 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:26.426841 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:26.427495 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:26.427726 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:26.432135 (+ 4409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:26.432142 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:26.432153 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:00:26.432192 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:26.436011 (+ 3819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:26.436825 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:26.436834 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:26.436840 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:26.437034 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:26.437458 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:26.437462 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:26.437464 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:26.438046 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:26.438232 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:26.438898 (+ 666us) server_negotiation.cc:299] Negotiation successful
0924 01:00:26.439285 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":430,"threads_started":1}
I20260924 01:00:26.440052 25557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:26.417391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:57241 (local address 127.8.64.130:43121)
0924 01:00:26.418069 (+ 678us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:26.418083 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:26.418134 (+ 51us) server_negotiation.cc:407] Connection header received
0924 01:00:26.420850 (+ 2716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:26.420861 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:26.421376 (+ 515us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:26.421778 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:26.423755 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:26.424910 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:26.426881 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:26.427485 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:26.427669 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:26.431909 (+ 4240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:26.431917 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:26.431931 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:00:26.431967 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:26.435278 (+ 3311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:26.436233 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:26.436243 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:26.436253 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:26.436451 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:26.436984 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:26.436988 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:26.436991 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:26.437689 (+ 698us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:26.437830 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:26.438876 (+ 1046us) server_negotiation.cc:299] Negotiation successful
0924 01:00:26.439303 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":328,"threads_started":1}
I20260924 01:00:26.441375 25558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:26.417745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.131:38205 (local address 127.8.64.129:56149)
0924 01:00:26.418397 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:26.418417 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:26.418548 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:26.422462 (+ 3914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:26.422467 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:26.423005 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0924 01:00:26.423554 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:26.423573 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:26.425120 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:26.425132 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:26.426692 (+ 1560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:26.426705 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:26.427742 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:26.427749 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:26.428006 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:26.428785 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 01:00:26.428812 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0924 01:00:26.431870 (+ 3058us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 01:00:26.436199 (+ 4329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:00:26.436204 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:00:26.436207 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:00:26.436685 (+ 478us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:00:26.437163 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:00:26.437168 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:00:26.437171 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:00:26.437346 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:00:26.438211 (+ 865us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 01:00:26.438215 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 01:00:26.438445 (+ 230us) client_negotiation.cc:769] Sending connection context
0924 01:00:26.438655 (+ 210us) client_negotiation.cc:240] Negotiation successful
0924 01:00:26.439302 (+ 647us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"mutex_wait_us":341,"spinlock_wait_cycles":19968,"thread_start_us":678,"threads_started":1}
W20260924 01:00:26.443715 25173 leader_election.cc:341] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ae77c9ce9b674ebbb2f57f0bcc293036 (127.8.64.130:43121): Not found: Tablet not found: 28c04e23639c4318a322edab1264552d
W20260924 01:00:26.444249 25173 leader_election.cc:341] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a1609a2bbb742e4b6722861bd63c4a8 (127.8.64.131:38205): Not found: Tablet not found: 28c04e23639c4318a322edab1264552d
I20260924 01:00:26.444438 25173 leader_election.cc:302] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4727a59d1809474dad8a93ba1bf545be; no voters: 4a1609a2bbb742e4b6722861bd63c4a8, ae77c9ce9b674ebbb2f57f0bcc293036
I20260924 01:00:26.445094 25555 raft_consensus.cc:2726] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 01:00:26.540014 25560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:26.539137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.131:40953)
0924 01:00:26.539624 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:26.539766 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":240,"threads_started":1}
I20260924 01:00:26.621699 25558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:26.621229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.129:44107)
0924 01:00:26.621448 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:26.621559 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260924 01:00:27.077025 25561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:27.076304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.130:50019)
0924 01:00:27.076772 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:27.076852 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":261,"threads_started":1}
I20260924 01:00:27.541669 25562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:27.541018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.131:51523)
0924 01:00:27.541434 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:27.541518 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":227,"threads_started":1}
I20260924 01:00:27.623440 25563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:27.622670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.129:55679)
0924 01:00:27.623188 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:27.623270 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":290,"threads_started":1}
I20260924 01:00:28.078765 25564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:28.077992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.130:56329)
0924 01:00:28.078518 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:28.078600 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":239,"threads_started":1}
I20260924 01:00:28.502696 25565 raft_consensus.cc:482] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 01:00:28.502897 25565 raft_consensus.cc:504] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4727a59d1809474dad8a93ba1bf545be" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37443 } } peers { permanent_uuid: "ae77c9ce9b674ebbb2f57f0bcc293036" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 43121 } } peers { permanent_uuid: "4a1609a2bbb742e4b6722861bd63c4a8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 38205 } }
I20260924 01:00:28.503484 25565 leader_election.cc:288] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ae77c9ce9b674ebbb2f57f0bcc293036 (127.8.64.130:43121), 4a1609a2bbb742e4b6722861bd63c4a8 (127.8.64.131:38205)
W20260924 01:00:28.504645 25173 leader_election.cc:341] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ae77c9ce9b674ebbb2f57f0bcc293036 (127.8.64.130:43121): Not found: Tablet not found: 28c04e23639c4318a322edab1264552d
W20260924 01:00:28.504916 25173 leader_election.cc:341] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a1609a2bbb742e4b6722861bd63c4a8 (127.8.64.131:38205): Not found: Tablet not found: 28c04e23639c4318a322edab1264552d
I20260924 01:00:28.505029 25173 leader_election.cc:302] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4727a59d1809474dad8a93ba1bf545be; no voters: 4a1609a2bbb742e4b6722861bd63c4a8, ae77c9ce9b674ebbb2f57f0bcc293036
I20260924 01:00:28.505293 25565 raft_consensus.cc:2726] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 01:00:28.543358 25566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:28.542685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.131:38577)
0924 01:00:28.543113 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:28.543199 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":218,"threads_started":1}
I20260924 01:00:28.625191 25567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:28.624385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.129:57993)
0924 01:00:28.624899 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:28.624992 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":250,"threads_started":1}
I20260924 01:00:29.080483 25568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:29.079795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.130:45999)
0924 01:00:29.080216 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:29.080300 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":206,"threads_started":1}
I20260924 01:00:29.545172 25569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:29.544424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.131:51601)
0924 01:00:29.544908 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:29.544995 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":240,"threads_started":1}
I20260924 01:00:29.627131 25570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:29.626245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.129:44047)
0924 01:00:29.626839 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:29.626929 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":257,"threads_started":1}
I20260924 01:00:30.082125 25571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:30.081463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.130:42919)
0924 01:00:30.081867 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:30.081944 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":219,"threads_started":1}
I20260924 01:00:30.546970 25572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:30.546256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.131:49237)
0924 01:00:30.546730 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:30.546807 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":235,"threads_started":1}
I20260924 01:00:30.628827 25573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:30.628043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.129:33151)
0924 01:00:30.628562 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:30.628657 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":232,"threads_started":1}
I20260924 01:00:31.031256 25574 raft_consensus.cc:482] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 01:00:31.031574 25574 raft_consensus.cc:504] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4727a59d1809474dad8a93ba1bf545be" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37443 } } peers { permanent_uuid: "ae77c9ce9b674ebbb2f57f0bcc293036" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 43121 } } peers { permanent_uuid: "4a1609a2bbb742e4b6722861bd63c4a8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 38205 } }
I20260924 01:00:31.032213 25574 leader_election.cc:288] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ae77c9ce9b674ebbb2f57f0bcc293036 (127.8.64.130:43121), 4a1609a2bbb742e4b6722861bd63c4a8 (127.8.64.131:38205)
W20260924 01:00:31.033161 25173 leader_election.cc:341] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ae77c9ce9b674ebbb2f57f0bcc293036 (127.8.64.130:43121): Not found: Tablet not found: 28c04e23639c4318a322edab1264552d
W20260924 01:00:31.033342 25173 leader_election.cc:341] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a1609a2bbb742e4b6722861bd63c4a8 (127.8.64.131:38205): Not found: Tablet not found: 28c04e23639c4318a322edab1264552d
I20260924 01:00:31.033432 25173 leader_election.cc:302] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4727a59d1809474dad8a93ba1bf545be; no voters: 4a1609a2bbb742e4b6722861bd63c4a8, ae77c9ce9b674ebbb2f57f0bcc293036
I20260924 01:00:31.033648 25574 raft_consensus.cc:2726] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 01:00:31.083774 25575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:31.083038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.130:38049)
0924 01:00:31.083513 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:31.083593 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":259,"threads_started":1}
I20260924 01:00:31.548686 25576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:31.547968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.131:52679)
0924 01:00:31.548413 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:31.548493 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":226,"threads_started":1}
I20260924 01:00:31.631088 25577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:31.629954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.129:59847)
0924 01:00:31.630704 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:31.630817 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":421,"threads_started":1}
I20260924 01:00:32.085522 25578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:32.084762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.130:52727)
0924 01:00:32.085248 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:32.085335 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":232,"threads_started":1}
I20260924 01:00:32.550580 25579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:32.549717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.131:50335)
0924 01:00:32.550245 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:32.550386 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":250,"threads_started":1}
I20260924 01:00:32.632699 25580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:32.631996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.129:46207)
0924 01:00:32.632437 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:32.632524 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":243,"threads_started":1}
I20260924 01:00:33.087340 25581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:33.086572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.130:50099)
0924 01:00:33.087046 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:33.087150 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":251,"threads_started":1}
I20260924 01:00:33.552171 25582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:33.551462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.131:33999)
0924 01:00:33.551896 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:33.551976 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":215,"threads_started":1}
I20260924 01:00:33.634650 25583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:33.633704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.129:44147)
0924 01:00:33.634352 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:33.634459 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":324,"threads_started":1}
I20260924 01:00:34.089039 25584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:34.088219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.130:39193)
0924 01:00:34.088765 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:34.088853 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":302,"threads_started":1}
I20260924 01:00:34.553848 25585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:34.553129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.131:33803)
0924 01:00:34.553608 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:34.553697 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":240,"threads_started":1}
I20260924 01:00:34.636498 25586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:34.635680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.129:58073)
0924 01:00:34.636252 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:34.636330 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":334,"threads_started":1}
I20260924 01:00:35.090987 25587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:35.090124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.130:39769)
0924 01:00:35.090724 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:35.090813 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":349,"threads_started":1}
I20260924 01:00:35.205286 25588 raft_consensus.cc:482] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 01:00:35.205476 25588 raft_consensus.cc:504] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4727a59d1809474dad8a93ba1bf545be" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37443 } } peers { permanent_uuid: "ae77c9ce9b674ebbb2f57f0bcc293036" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 43121 } } peers { permanent_uuid: "4a1609a2bbb742e4b6722861bd63c4a8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 38205 } }
I20260924 01:00:35.206061 25588 leader_election.cc:288] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ae77c9ce9b674ebbb2f57f0bcc293036 (127.8.64.130:43121), 4a1609a2bbb742e4b6722861bd63c4a8 (127.8.64.131:38205)
W20260924 01:00:35.207123 25173 leader_election.cc:341] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a1609a2bbb742e4b6722861bd63c4a8 (127.8.64.131:38205): Not found: Tablet not found: 28c04e23639c4318a322edab1264552d
W20260924 01:00:35.207345 25173 leader_election.cc:341] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ae77c9ce9b674ebbb2f57f0bcc293036 (127.8.64.130:43121): Not found: Tablet not found: 28c04e23639c4318a322edab1264552d
I20260924 01:00:35.207424 25173 leader_election.cc:302] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4727a59d1809474dad8a93ba1bf545be; no voters: 4a1609a2bbb742e4b6722861bd63c4a8, ae77c9ce9b674ebbb2f57f0bcc293036
I20260924 01:00:35.207731 25588 raft_consensus.cc:2726] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 01:00:35.555611 25589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:35.554871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.131:34139)
0924 01:00:35.555350 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:35.555432 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":272,"threads_started":1}
I20260924 01:00:35.638216 25590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:35.637385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.129:51985)
0924 01:00:35.637896 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:35.638025 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":251,"threads_started":1}
I20260924 01:00:36.092736 25591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:36.091975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.130:49813)
0924 01:00:36.092484 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:36.092569 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":238,"threads_started":1}
I20260924 01:00:36.557397 25592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:36.556667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.131:49445)
0924 01:00:36.557151 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:36.557240 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":251,"threads_started":1}
I20260924 01:00:36.639761 25593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:36.639097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.129:55527)
0924 01:00:36.639521 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:36.639599 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":210,"threads_started":1}
I20260924 01:00:37.094549 25594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:37.093696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.130:59899)
0924 01:00:37.094238 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:37.094380 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":324,"threads_started":1}
I20260924 01:00:37.433615 25595 raft_consensus.cc:482] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 01:00:37.433851 25595 raft_consensus.cc:504] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4727a59d1809474dad8a93ba1bf545be" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37443 } } peers { permanent_uuid: "ae77c9ce9b674ebbb2f57f0bcc293036" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 43121 } } peers { permanent_uuid: "4a1609a2bbb742e4b6722861bd63c4a8" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 38205 } }
I20260924 01:00:37.434547 25595 leader_election.cc:288] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ae77c9ce9b674ebbb2f57f0bcc293036 (127.8.64.130:43121), 4a1609a2bbb742e4b6722861bd63c4a8 (127.8.64.131:38205)
W20260924 01:00:37.435468 25173 leader_election.cc:341] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ae77c9ce9b674ebbb2f57f0bcc293036 (127.8.64.130:43121): Not found: Tablet not found: 28c04e23639c4318a322edab1264552d
W20260924 01:00:37.435671 25173 leader_election.cc:341] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a1609a2bbb742e4b6722861bd63c4a8 (127.8.64.131:38205): Not found: Tablet not found: 28c04e23639c4318a322edab1264552d
I20260924 01:00:37.435787 25173 leader_election.cc:302] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4727a59d1809474dad8a93ba1bf545be; no voters: 4a1609a2bbb742e4b6722861bd63c4a8, ae77c9ce9b674ebbb2f57f0bcc293036
I20260924 01:00:37.436031 25595 raft_consensus.cc:2726] T 28c04e23639c4318a322edab1264552d P 4727a59d1809474dad8a93ba1bf545be [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 01:00:37.559284 25596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:37.558509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.131:58031)
0924 01:00:37.559022 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:37.559105 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":283,"threads_started":1}
I20260924 01:00:37.641302 25597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:37.640574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.129:44343)
0924 01:00:37.640979 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:37.641076 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":212,"threads_started":1}
I20260924 01:00:38.096213 25598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:38.095496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.130:34159)
0924 01:00:38.095951 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:38.096042 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":242,"threads_started":1}
I20260924 01:00:38.561336 25599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:38.560339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.131:34219)
0924 01:00:38.560963 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:38.561165 (+ 202us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":346,"threads_started":1}
I20260924 01:00:38.642928 25600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:38.642217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.129:51201)
0924 01:00:38.642658 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:38.642747 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":236,"threads_started":1}
I20260924 01:00:39.098066 25601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:39.097214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.130:59669)
0924 01:00:39.097762 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:39.097851 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":301,"threads_started":1}
I20260924 01:00:39.563203 25602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:39.562488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.131:43789)
0924 01:00:39.562926 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:39.563009 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":220,"threads_started":1}
I20260924 01:00:39.644936 25603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:39.644217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.129:35557)
0924 01:00:39.644662 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:39.644756 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":226,"threads_started":1}
I20260924 01:00:40.100036 25604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:40.099272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.130:60099)
0924 01:00:40.099758 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:40.099877 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":253,"threads_started":1}
I20260924 01:00:40.544670 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 25116
I20260924 01:00:40.560530 25605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:40.545955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58854 (local address 127.8.64.129:37443)
0924 01:00:40.546683 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:40.546696 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:40.546734 (+ 38us) server_negotiation.cc:407] Connection header received
0924 01:00:40.546819 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:40.546832 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:40.547139 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:40.547277 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:40.548730 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:40.549716 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:40.551072 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:40.551679 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:40.551789 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:40.554492 (+ 2703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:40.554506 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:40.554522 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:00:40.554563 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:40.557681 (+ 3118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:40.558408 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:40.558416 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:40.558418 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:40.558592 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:40.559043 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:40.559046 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:40.559048 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:40.559385 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:40.559657 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:40.559908 (+ 251us) server_negotiation.cc:299] Negotiation successful
0924 01:00:40.560115 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":337,"threads_started":1}
I20260924 01:00:40.561923 25282 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 01:00:40.565383 25607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:40.564575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.131:37061)
0924 01:00:40.565065 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:40.565188 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":253,"threads_started":1}
I20260924 01:00:40.650341 25608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:40.649347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42983 (local address 127.8.64.129:57261)
0924 01:00:40.649879 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:40.650005 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.190:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":244,"threads_started":1}
I20260924 01:00:40.650656 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 25116
I20260924 01:00:40.668215 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 25290
I20260924 01:00:40.681088 25609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:40.668940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37898 (local address 127.8.64.130:43121)
0924 01:00:40.669418 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:40.669423 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:40.669439 (+ 16us) server_negotiation.cc:407] Connection header received
0924 01:00:40.669494 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:40.669497 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:40.669562 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:40.669653 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:40.670982 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:40.671731 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:40.673036 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:40.673466 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:40.673573 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:40.676106 (+ 2533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:40.676126 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:40.676129 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:00:40.676160 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:40.678754 (+ 2594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:40.679339 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:40.679343 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:40.679345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:40.679467 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:40.679829 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:40.679833 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:40.679835 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:40.680107 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:40.680310 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:40.680625 (+ 315us) server_negotiation.cc:299] Negotiation successful
0924 01:00:40.680786 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":285,"threads_started":1}
I20260924 01:00:40.681739 25413 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 01:00:40.766535 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 25290
I20260924 01:00:40.784299 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 25421
I20260924 01:00:40.797657 25611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:40.785199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35504 (local address 127.8.64.131:38205)
0924 01:00:40.785817 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:40.785822 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:40.785840 (+ 18us) server_negotiation.cc:407] Connection header received
0924 01:00:40.785897 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:40.785901 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:40.785970 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:40.786064 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:40.787470 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:40.788264 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:40.789467 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:40.789868 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:40.789959 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:40.792733 (+ 2774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:40.792738 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:40.792741 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:00:40.792773 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:40.795343 (+ 2570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:40.795945 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:40.795948 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:40.795951 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:40.796070 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:40.796423 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:40.796426 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:40.796428 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:40.796686 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:40.796880 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:40.797184 (+ 304us) server_negotiation.cc:299] Negotiation successful
0924 01:00:40.797353 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":351,"threads_started":1}
I20260924 01:00:40.798400 25544 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 01:00:40.881531 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 25421
2026-09-24T01:00:40Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64384 ms)
[----------] 30 tests from SecurityITest (297747 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 01:00:40 dist-test-slave-mmhs krb5kdc[25615](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 01:00:40 dist-test-slave-mmhs krb5kdc[25615](info): set up 2 sockets
Sep 24 01:00:40 dist-test-slave-mmhs krb5kdc[25615](info): commencing operation
krb5kdc: starting...
W20260924 01:00:41.996182 8450 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.005s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Sep 24 01:00:42 dist-test-slave-mmhs krb5kdc[25615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T01:00:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T01:00:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.64.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
I20260924 01:00:42.255637 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:41597
--webserver_interface=127.8.64.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:37167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:41597
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 01:00:42.584858 25628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:00:42.585166 25628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:00:42.585258 25628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:00:42.592418 25628 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 01:00:42.592535 25628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:00:42.592589 25628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:00:42.592641 25628 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 01:00:42.592686 25628 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 01:00:42.602409 25628 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:37167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:41597
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:41597
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:00:42.603634 25628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:00:42.604863 25628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:00:42.617237 25634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:42.617316 25633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:00:42.617542 25628 server_base.cc:1034] running on GCE node
W20260924 01:00:42.617291 25636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:00:42.618780 25628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:00:42.620307 25628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:00:42.621587 25628 hybrid_clock.cc:648] HybridClock initialized: now 1790211642621533 us; error 44 us; skew 500 ppm
I20260924 01:00:42.622006 25628 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 01:00:42 dist-test-slave-mmhs krb5kdc[25615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211642, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 01:00:42.632942 25628 init.cc:377] Logged in from keytab as kudu/127.8.64.190@KRBTEST.COM (short username kudu)
I20260924 01:00:42.634585 25628 webserver.cc:460] Webserver started at http://127.8.64.190:43889/ using document root <none> and password file <none>
I20260924 01:00:42.635245 25628 fs_manager.cc:359] Metadata directory not provided
I20260924 01:00:42.635349 25628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:00:42.635617 25628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:00:42.638692 25628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "f80d379c1ee84cc09ba87b5ea0615bb4"
format_stamp: "Formatted at 2026-09-24 01:00:42 on dist-test-slave-mmhs"
server_key: "3a0a5947a9ae38b51d2d9f94608267c8"
server_key_iv: "dbdcbfad258cdc15a882dde08829551d"
server_key_version: "encryptionkey@0"
I20260924 01:00:42.639463 25628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "f80d379c1ee84cc09ba87b5ea0615bb4"
format_stamp: "Formatted at 2026-09-24 01:00:42 on dist-test-slave-mmhs"
server_key: "3a0a5947a9ae38b51d2d9f94608267c8"
server_key_iv: "dbdcbfad258cdc15a882dde08829551d"
server_key_version: "encryptionkey@0"
I20260924 01:00:42.644780 25628 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.005s
I20260924 01:00:42.648270 25643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:42.649528 25628 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260924 01:00:42.649710 25628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "f80d379c1ee84cc09ba87b5ea0615bb4"
format_stamp: "Formatted at 2026-09-24 01:00:42 on dist-test-slave-mmhs"
server_key: "3a0a5947a9ae38b51d2d9f94608267c8"
server_key_iv: "dbdcbfad258cdc15a882dde08829551d"
server_key_version: "encryptionkey@0"
I20260924 01:00:42.649907 25628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:00:42.671821 25628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:00:42.675796 25628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:00:42.676092 25628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:00:42.695093 25628 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.190:41597
I20260924 01:00:42.695144 25694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.190:41597 every 8 connection(s)
I20260924 01:00:42.696532 25628 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 01:00:42.700407 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 25628
I20260924 01:00:42.700541 25695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:00:42.700649 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 01:00:42.701462 8450 external_mini_cluster.cc:1428] Setting key 1020736d8384129f3707b5be4aa84de2
I20260924 01:00:42.715144 25695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f80d379c1ee84cc09ba87b5ea0615bb4: Bootstrap starting.
Sep 24 01:00:42 dist-test-slave-mmhs krb5kdc[25615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 01:00:42.718837 25695 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f80d379c1ee84cc09ba87b5ea0615bb4: Neither blocks nor log segments found. Creating new log.
I20260924 01:00:42.720091 25695 log.cc:824] T 00000000000000000000000000000000 P f80d379c1ee84cc09ba87b5ea0615bb4: Log is configured to *not* fsync() on all Append() calls
I20260924 01:00:42.723037 25695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f80d379c1ee84cc09ba87b5ea0615bb4: No bootstrap required, opened a new log
I20260924 01:00:42.725997 25698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:42.703685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43246 (local address 127.8.64.190:41597)
0924 01:00:42.704530 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:42.704542 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:42.704582 (+ 40us) server_negotiation.cc:407] Connection header received
0924 01:00:42.706960 (+ 2378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:42.706987 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:42.707400 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:42.707735 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:42.709647 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:42.710972 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:42.712919 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:42.713565 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:42.713708 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:42.718423 (+ 4715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:42.718432 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:42.718447 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:00:42.718484 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:42.721812 (+ 3328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:42.722622 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:42.722629 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:42.722635 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:42.722801 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:42.723215 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:42.723220 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:42.723225 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:42.723743 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:42.724078 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:42.724537 (+ 459us) server_negotiation.cc:299] Negotiation successful
0924 01:00:42.724996 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":429,"threads_started":1}
I20260924 01:00:42.731376 25695 raft_consensus.cc:348] T 00000000000000000000000000000000 P f80d379c1ee84cc09ba87b5ea0615bb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f80d379c1ee84cc09ba87b5ea0615bb4" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 41597 } }
I20260924 01:00:42.731873 25695 raft_consensus.cc:374] T 00000000000000000000000000000000 P f80d379c1ee84cc09ba87b5ea0615bb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 01:00:42.731987 25695 raft_consensus.cc:729] T 00000000000000000000000000000000 P f80d379c1ee84cc09ba87b5ea0615bb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f80d379c1ee84cc09ba87b5ea0615bb4, State: Initialized, Role: FOLLOWER
I20260924 01:00:42.732611 25695 consensus_queue.cc:260] T 00000000000000000000000000000000 P f80d379c1ee84cc09ba87b5ea0615bb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f80d379c1ee84cc09ba87b5ea0615bb4" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 41597 } }
I20260924 01:00:42.732818 25695 raft_consensus.cc:388] T 00000000000000000000000000000000 P f80d379c1ee84cc09ba87b5ea0615bb4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 01:00:42.732916 25695 raft_consensus.cc:482] T 00000000000000000000000000000000 P f80d379c1ee84cc09ba87b5ea0615bb4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 01:00:42.733031 25695 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f80d379c1ee84cc09ba87b5ea0615bb4 [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:00:42.735169 25695 raft_consensus.cc:504] T 00000000000000000000000000000000 P f80d379c1ee84cc09ba87b5ea0615bb4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f80d379c1ee84cc09ba87b5ea0615bb4" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 41597 } }
I20260924 01:00:42.735600 25695 leader_election.cc:302] T 00000000000000000000000000000000 P f80d379c1ee84cc09ba87b5ea0615bb4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f80d379c1ee84cc09ba87b5ea0615bb4; no voters:
I20260924 01:00:42.736277 25695 leader_election.cc:288] T 00000000000000000000000000000000 P f80d379c1ee84cc09ba87b5ea0615bb4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 01:00:42.736405 25700 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f80d379c1ee84cc09ba87b5ea0615bb4 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 01:00:42.737066 25700 raft_consensus.cc:686] T 00000000000000000000000000000000 P f80d379c1ee84cc09ba87b5ea0615bb4 [term 1 LEADER]: Becoming Leader. State: Replica: f80d379c1ee84cc09ba87b5ea0615bb4, State: Running, Role: LEADER
I20260924 01:00:42.737655 25700 consensus_queue.cc:237] T 00000000000000000000000000000000 P f80d379c1ee84cc09ba87b5ea0615bb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f80d379c1ee84cc09ba87b5ea0615bb4" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 41597 } }
I20260924 01:00:42.738410 25695 sys_catalog.cc:564] T 00000000000000000000000000000000 P f80d379c1ee84cc09ba87b5ea0615bb4 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 01:00:42.740556 25701 sys_catalog.cc:455] T 00000000000000000000000000000000 P f80d379c1ee84cc09ba87b5ea0615bb4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f80d379c1ee84cc09ba87b5ea0615bb4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f80d379c1ee84cc09ba87b5ea0615bb4" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 41597 } } }
I20260924 01:00:42.741019 25701 sys_catalog.cc:458] T 00000000000000000000000000000000 P f80d379c1ee84cc09ba87b5ea0615bb4 [sys.catalog]: This master's current role is: LEADER
I20260924 01:00:42.741961 25702 sys_catalog.cc:455] T 00000000000000000000000000000000 P f80d379c1ee84cc09ba87b5ea0615bb4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f80d379c1ee84cc09ba87b5ea0615bb4. Latest consensus state: current_term: 1 leader_uuid: "f80d379c1ee84cc09ba87b5ea0615bb4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f80d379c1ee84cc09ba87b5ea0615bb4" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 41597 } } }
I20260924 01:00:42.742229 25709 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 01:00:42.742360 25702 sys_catalog.cc:458] T 00000000000000000000000000000000 P f80d379c1ee84cc09ba87b5ea0615bb4 [sys.catalog]: This master's current role is: LEADER
I20260924 01:00:42.750057 25709 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 01:00:42.759924 25709 catalog_manager.cc:1347] Generated new cluster ID: 49bba151a2e44e00ad3d1363da29352c
I20260924 01:00:42.760051 25709 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 01:00:42.778494 25709 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 01:00:42.780089 25709 catalog_manager.cc:1504] Loading token signing keys...
I20260924 01:00:42.798007 25709 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f80d379c1ee84cc09ba87b5ea0615bb4: Generated new TSK 0
I20260924 01:00:42.799069 25709 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.64.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab.
Entry for principal kudu/127.8.64.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab.
I20260924 01:00:42.870008 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.129:0
--local_ip_for_outbound_sockets=127.8.64.129
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:41597
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab
--principal=kudu/127.8.64.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:37167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 01:00:43.204641 25721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:00:43.204910 25721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:00:43.205005 25721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:00:43.212387 25721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:00:43.212487 25721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:00:43.212597 25721 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.129
I20260924 01:00:43.222134 25721 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:37167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab
--principal=kudu/127.8.64.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:41597
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.129
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:00:43.223395 25721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:00:43.224706 25721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:00:43.237906 25729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:43.237993 25726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:43.237964 25727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:00:43.243872 25721 server_base.cc:1034] running on GCE node
I20260924 01:00:43.244510 25721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:00:43.245551 25721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:00:43.246737 25721 hybrid_clock.cc:648] HybridClock initialized: now 1790211643246693 us; error 32 us; skew 500 ppm
I20260924 01:00:43.247061 25721 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 01:00:43 dist-test-slave-mmhs krb5kdc[25615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211643, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 01:00:43.250404 25721 init.cc:377] Logged in from keytab as kudu/127.8.64.129@KRBTEST.COM (short username kudu)
I20260924 01:00:43.251935 25721 webserver.cc:460] Webserver started at http://127.8.64.129:39687/ using document root <none> and password file <none>
I20260924 01:00:43.252642 25721 fs_manager.cc:359] Metadata directory not provided
I20260924 01:00:43.252769 25721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:00:43.253106 25721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:00:43.256062 25721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-0/data/instance:
uuid: "52e8e0bbed514465a08aef517ae553c5"
format_stamp: "Formatted at 2026-09-24 01:00:43 on dist-test-slave-mmhs"
server_key: "40b9c9fbd11ba62891e35b463962f075"
server_key_iv: "56aa64159b8e34c295da2783f4530b1f"
server_key_version: "encryptionkey@0"
I20260924 01:00:43.256843 25721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance:
uuid: "52e8e0bbed514465a08aef517ae553c5"
format_stamp: "Formatted at 2026-09-24 01:00:43 on dist-test-slave-mmhs"
server_key: "40b9c9fbd11ba62891e35b463962f075"
server_key_iv: "56aa64159b8e34c295da2783f4530b1f"
server_key_version: "encryptionkey@0"
I20260924 01:00:43.261760 25721 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260924 01:00:43.265172 25736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:43.266562 25721 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 01:00:43.266716 25721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-0/wal
uuid: "52e8e0bbed514465a08aef517ae553c5"
format_stamp: "Formatted at 2026-09-24 01:00:43 on dist-test-slave-mmhs"
server_key: "40b9c9fbd11ba62891e35b463962f075"
server_key_iv: "56aa64159b8e34c295da2783f4530b1f"
server_key_version: "encryptionkey@0"
I20260924 01:00:43.266903 25721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:00:43.293493 25721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:00:43.298147 25721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:00:43.298493 25721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:00:43.299433 25721 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 01:00:43.300805 25721 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 01:00:43.300879 25721 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:43.301060 25721 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 01:00:43.301113 25721 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:43.334818 25721 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.129:37765
I20260924 01:00:43.334909 25848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.129:37765 every 8 connection(s)
I20260924 01:00:43.336138 25721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
I20260924 01:00:43.344903 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 25721
I20260924 01:00:43.345124 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance
I20260924 01:00:43.345768 8450 external_mini_cluster.cc:1428] Setting key 6a93e3d1fb318c02bbc9716c1348da5f
Sep 24 01:00:43 dist-test-slave-mmhs krb5kdc[25615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211643, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 01:00:43.357045 25851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:43.338751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:38123 (local address 127.8.64.190:41597)
0924 01:00:43.339271 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:43.339275 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:43.340226 (+ 951us) server_negotiation.cc:407] Connection header received
0924 01:00:43.342331 (+ 2105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:43.342337 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:43.342428 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:43.342571 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:43.344051 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:43.345072 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:43.346920 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:43.347361 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:43.347502 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:43.351174 (+ 3672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:43.351179 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:43.351183 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:00:43.351212 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:43.353932 (+ 2720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:43.354784 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:43.354794 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:43.354796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:43.354910 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:43.355331 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:43.355336 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:43.355339 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:43.355603 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:43.355758 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:43.356464 (+ 706us) server_negotiation.cc:299] Negotiation successful
0924 01:00:43.356685 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":278,"threads_started":1}
I20260924 01:00:43.357987 25852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:43.339327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:41597 (local address 127.8.64.129:38123)
0924 01:00:43.340073 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:43.340120 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:43.342031 (+ 1911us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:43.342807 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:43.342815 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:43.343319 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0924 01:00:43.343873 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:43.343891 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:43.345254 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:43.345267 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:43.346763 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:43.346780 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:43.347614 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:43.347621 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:43.347874 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:43.348602 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 01:00:43.348623 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0924 01:00:43.350951 (+ 2328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 01:00:43.354105 (+ 3154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:00:43.354115 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:00:43.354131 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:00:43.354638 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:00:43.355018 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:00:43.355026 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:00:43.355030 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:00:43.355211 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:00:43.355732 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 01:00:43.355740 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 01:00:43.356066 (+ 326us) client_negotiation.cc:769] Sending connection context
0924 01:00:43.356345 (+ 279us) client_negotiation.cc:240] Negotiation successful
0924 01:00:43.356876 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":278,"threads_started":1}
I20260924 01:00:43.359654 25849 heartbeater.cc:347] Connected to a master server at 127.8.64.190:41597
I20260924 01:00:43.359961 25849 heartbeater.cc:464] Registering TS with master...
I20260924 01:00:43.360646 25849 heartbeater.cc:511] Master 127.8.64.190:41597 requested a full tablet report, sending...
I20260924 01:00:43.362947 25660 ts_manager.cc:194] Registered new tserver with Master: 52e8e0bbed514465a08aef517ae553c5 (127.8.64.129:37765)
I20260924 01:00:43.365130 25660 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.129@KRBTEST.COM'} at 127.8.64.129:38123
WARNING: no policy specified for kudu/127.8.64.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab.
Entry for principal kudu/127.8.64.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab.
I20260924 01:00:43.406978 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.130:0
--local_ip_for_outbound_sockets=127.8.64.130
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:41597
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab
--principal=kudu/127.8.64.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:37167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 01:00:43.733886 25855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:00:43.734174 25855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:00:43.734266 25855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:00:43.741626 25855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:00:43.741734 25855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:00:43.741845 25855 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.130
I20260924 01:00:43.751888 25855 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:37167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab
--principal=kudu/127.8.64.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:41597
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.130
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:00:43.753521 25855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:00:43.755126 25855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:00:43.769093 25863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:43.769106 25860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:43.769106 25861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:00:43.770057 25855 server_base.cc:1034] running on GCE node
I20260924 01:00:43.770876 25855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:00:43.771840 25855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:00:43.773118 25855 hybrid_clock.cc:648] HybridClock initialized: now 1790211643773049 us; error 52 us; skew 500 ppm
I20260924 01:00:43.773465 25855 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 01:00:43 dist-test-slave-mmhs krb5kdc[25615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211643, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 01:00:43.776721 25855 init.cc:377] Logged in from keytab as kudu/127.8.64.130@KRBTEST.COM (short username kudu)
I20260924 01:00:43.778136 25855 webserver.cc:460] Webserver started at http://127.8.64.130:42577/ using document root <none> and password file <none>
I20260924 01:00:43.778889 25855 fs_manager.cc:359] Metadata directory not provided
I20260924 01:00:43.779034 25855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:00:43.779345 25855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:00:43.782285 25855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-1/data/instance:
uuid: "8a37b2cb7a0f486d901c9020068772fb"
format_stamp: "Formatted at 2026-09-24 01:00:43 on dist-test-slave-mmhs"
server_key: "e9520ee0fbd7d6cdb210f69e5f6e1f84"
server_key_iv: "d23beed4082e04e9de8fb259a5e8f51e"
server_key_version: "encryptionkey@0"
I20260924 01:00:43.783128 25855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance:
uuid: "8a37b2cb7a0f486d901c9020068772fb"
format_stamp: "Formatted at 2026-09-24 01:00:43 on dist-test-slave-mmhs"
server_key: "e9520ee0fbd7d6cdb210f69e5f6e1f84"
server_key_iv: "d23beed4082e04e9de8fb259a5e8f51e"
server_key_version: "encryptionkey@0"
I20260924 01:00:43.788129 25855 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260924 01:00:43.791723 25870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:43.793124 25855 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260924 01:00:43.793320 25855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-1/wal
uuid: "8a37b2cb7a0f486d901c9020068772fb"
format_stamp: "Formatted at 2026-09-24 01:00:43 on dist-test-slave-mmhs"
server_key: "e9520ee0fbd7d6cdb210f69e5f6e1f84"
server_key_iv: "d23beed4082e04e9de8fb259a5e8f51e"
server_key_version: "encryptionkey@0"
I20260924 01:00:43.793545 25855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:00:43.810534 25855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:00:43.814563 25855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:00:43.814838 25855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:00:43.815764 25855 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 01:00:43.817054 25855 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 01:00:43.817157 25855 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:43.817302 25855 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 01:00:43.817373 25855 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:43.850891 25855 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.130:37165
I20260924 01:00:43.850962 25982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.130:37165 every 8 connection(s)
I20260924 01:00:43.852202 25855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
I20260924 01:00:43.862628 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 25855
I20260924 01:00:43.862852 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance
I20260924 01:00:43.863487 8450 external_mini_cluster.cc:1428] Setting key c37824cad1fdfce7983adcb4754435ae
Sep 24 01:00:43 dist-test-slave-mmhs krb5kdc[25615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211643, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.130@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 01:00:43.872619 25851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:43.855006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:58821 (local address 127.8.64.190:41597)
0924 01:00:43.855202 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:43.855207 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:43.856298 (+ 1091us) server_negotiation.cc:407] Connection header received
0924 01:00:43.858122 (+ 1824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:43.858127 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:43.858199 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:43.858345 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:43.859853 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:43.860586 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:43.862240 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:43.862729 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:43.862867 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:43.866722 (+ 3855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:43.866727 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:43.866730 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:00:43.866760 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:43.869344 (+ 2584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:43.870209 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:43.870219 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:43.870221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:43.870412 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:43.870831 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:43.870838 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:43.870841 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:43.871198 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:43.871353 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:43.872047 (+ 694us) server_negotiation.cc:299] Negotiation successful
0924 01:00:43.872276 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260924 01:00:43.873605 25985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:43.855387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:41597 (local address 127.8.64.130:58821)
0924 01:00:43.856125 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:43.856163 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:43.857851 (+ 1688us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:43.858617 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:43.858626 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:43.859133 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0924 01:00:43.859697 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:43.859714 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:43.860764 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:43.860768 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:43.862071 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:43.862084 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:43.863005 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:43.863015 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:43.863335 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:43.864046 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 01:00:43.864074 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0924 01:00:43.866493 (+ 2419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 01:00:43.869504 (+ 3011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:00:43.869513 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:00:43.869531 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:00:43.870089 (+ 558us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:00:43.870525 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:00:43.870530 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:00:43.870534 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:00:43.870720 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:00:43.871325 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 01:00:43.871333 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 01:00:43.871683 (+ 350us) client_negotiation.cc:769] Sending connection context
0924 01:00:43.871951 (+ 268us) client_negotiation.cc:240] Negotiation successful
0924 01:00:43.872473 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":280,"threads_started":1}
I20260924 01:00:43.875349 25983 heartbeater.cc:347] Connected to a master server at 127.8.64.190:41597
I20260924 01:00:43.875684 25983 heartbeater.cc:464] Registering TS with master...
I20260924 01:00:43.876572 25983 heartbeater.cc:511] Master 127.8.64.190:41597 requested a full tablet report, sending...
I20260924 01:00:43.878352 25660 ts_manager.cc:194] Registered new tserver with Master: 8a37b2cb7a0f486d901c9020068772fb (127.8.64.130:37165)
I20260924 01:00:43.879213 25660 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.130@KRBTEST.COM'} at 127.8.64.130:58821
WARNING: no policy specified for kudu/127.8.64.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab.
Entry for principal kudu/127.8.64.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab.
I20260924 01:00:43.923316 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.131:0
--local_ip_for_outbound_sockets=127.8.64.131
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:41597
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab
--principal=kudu/127.8.64.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:37167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 01:00:44.252283 25988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:00:44.252558 25988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:00:44.252651 25988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:00:44.259990 25988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:00:44.260094 25988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:00:44.260201 25988 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.131
I20260924 01:00:44.269537 25988 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:37167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab
--principal=kudu/127.8.64.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:41597
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.131
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:00:44.270771 25988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:00:44.272044 25988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:00:44.284796 25994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:44.284839 25996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:44.284873 25993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:00:44.285472 25988 server_base.cc:1034] running on GCE node
I20260924 01:00:44.286119 25988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:00:44.287181 25988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:00:44.288401 25988 hybrid_clock.cc:648] HybridClock initialized: now 1790211644288350 us; error 43 us; skew 500 ppm
I20260924 01:00:44.288715 25988 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 01:00:44 dist-test-slave-mmhs krb5kdc[25615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211644, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 01:00:44.292009 25988 init.cc:377] Logged in from keytab as kudu/127.8.64.131@KRBTEST.COM (short username kudu)
I20260924 01:00:44.293373 25988 webserver.cc:460] Webserver started at http://127.8.64.131:35777/ using document root <none> and password file <none>
I20260924 01:00:44.294061 25988 fs_manager.cc:359] Metadata directory not provided
I20260924 01:00:44.294203 25988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:00:44.294548 25988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:00:44.297490 25988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-2/data/instance:
uuid: "27921d144ddf4cffa3b14b3b704ba6bb"
format_stamp: "Formatted at 2026-09-24 01:00:44 on dist-test-slave-mmhs"
server_key: "838cc96ccf212a20b126f907a9bf91c2"
server_key_iv: "2ebc905011029424ef4fc81327b2a0ff"
server_key_version: "encryptionkey@0"
I20260924 01:00:44.298394 25988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance:
uuid: "27921d144ddf4cffa3b14b3b704ba6bb"
format_stamp: "Formatted at 2026-09-24 01:00:44 on dist-test-slave-mmhs"
server_key: "838cc96ccf212a20b126f907a9bf91c2"
server_key_iv: "2ebc905011029424ef4fc81327b2a0ff"
server_key_version: "encryptionkey@0"
I20260924 01:00:44.303340 25988 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260924 01:00:44.306735 26003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:44.307962 25988 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260924 01:00:44.308142 25988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-2/wal
uuid: "27921d144ddf4cffa3b14b3b704ba6bb"
format_stamp: "Formatted at 2026-09-24 01:00:44 on dist-test-slave-mmhs"
server_key: "838cc96ccf212a20b126f907a9bf91c2"
server_key_iv: "2ebc905011029424ef4fc81327b2a0ff"
server_key_version: "encryptionkey@0"
I20260924 01:00:44.308338 25988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:00:44.321033 25988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:00:44.325198 25988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:00:44.325495 25988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:00:44.326432 25988 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 01:00:44.327713 25988 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 01:00:44.327816 25988 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:44.327966 25988 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 01:00:44.328043 25988 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:44.362056 25988 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.131:37681
I20260924 01:00:44.362097 26115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.131:37681 every 8 connection(s)
I20260924 01:00:44.363368 25988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
I20260924 01:00:44.368150 25849 heartbeater.cc:503] Master 127.8.64.190:41597 was elected leader, sending a full tablet report...
I20260924 01:00:44.368968 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 25988
I20260924 01:00:44.369138 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance
I20260924 01:00:44.369711 8450 external_mini_cluster.cc:1428] Setting key a9a6e346e50b000a9b0cd32d8395bbe8
Sep 24 01:00:44 dist-test-slave-mmhs krb5kdc[25615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211644, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.131@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 01:00:44.384451 25851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:44.366088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:55655 (local address 127.8.64.190:41597)
0924 01:00:44.366316 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:44.366323 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:44.367508 (+ 1185us) server_negotiation.cc:407] Connection header received
0924 01:00:44.369992 (+ 2484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:44.369998 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:44.370079 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:44.370207 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:44.372195 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:44.373119 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:44.374695 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:44.375101 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:44.375236 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:44.378622 (+ 3386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:44.378627 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:44.378631 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:00:44.378662 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:44.381307 (+ 2645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:44.382005 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:44.382009 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:44.382011 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:44.382135 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:44.382594 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:44.382597 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:44.382599 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:44.382846 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:44.382973 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:44.383869 (+ 896us) server_negotiation.cc:299] Negotiation successful
0924 01:00:44.384100 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260924 01:00:44.385509 26118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:44.366625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:41597 (local address 127.8.64.131:55655)
0924 01:00:44.367345 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:44.367388 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:44.369692 (+ 2304us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:44.370510 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:44.370518 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:44.371230 (+ 712us) client_negotiation.cc:189] Negotiated authn=SASL
0924 01:00:44.371972 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:44.372008 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:44.373245 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:44.373249 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:44.374544 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:44.374556 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:44.375317 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:44.375327 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:44.375557 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:44.376220 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 01:00:44.376241 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0924 01:00:44.378396 (+ 2155us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 01:00:44.381448 (+ 3052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:00:44.381454 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:00:44.381467 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:00:44.381894 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:00:44.382227 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:00:44.382230 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:00:44.382232 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:00:44.382490 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:00:44.382971 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 01:00:44.382978 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 01:00:44.383367 (+ 389us) client_negotiation.cc:769] Sending connection context
0924 01:00:44.383745 (+ 378us) client_negotiation.cc:240] Negotiation successful
0924 01:00:44.384252 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":308,"threads_started":1}
I20260924 01:00:44.386893 26116 heartbeater.cc:347] Connected to a master server at 127.8.64.190:41597
I20260924 01:00:44.387203 26116 heartbeater.cc:464] Registering TS with master...
I20260924 01:00:44.387919 26116 heartbeater.cc:511] Master 127.8.64.190:41597 requested a full tablet report, sending...
I20260924 01:00:44.389261 25660 ts_manager.cc:194] Registered new tserver with Master: 27921d144ddf4cffa3b14b3b704ba6bb (127.8.64.131:37681)
I20260924 01:00:44.390079 25660 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.131@KRBTEST.COM'} at 127.8.64.131:55655
I20260924 01:00:44.397544 8450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 01:00:44.416431 25851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:44.402086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52972 (local address 127.8.64.190:41597)
0924 01:00:44.402328 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:44.402336 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:44.402778 (+ 442us) server_negotiation.cc:407] Connection header received
0924 01:00:44.402910 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:44.402914 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:44.402976 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:44.403060 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:44.404526 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:44.405217 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:44.406797 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:44.407227 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:44.407322 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:44.410228 (+ 2906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:44.410234 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:44.410257 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:00:44.410329 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:44.413339 (+ 3010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:44.414120 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:44.414126 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:44.414130 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:44.414287 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:44.414763 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:44.414770 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:44.414774 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:44.415151 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:44.415434 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:44.415821 (+ 387us) server_negotiation.cc:299] Negotiation successful
0924 01:00:44.416067 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260924 01:00:44.420483 25660 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52972:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 01:00:44.427950 25660 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 01:00:44.456676 26128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:44.441154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:37765 (local address 127.0.0.1:49504)
0924 01:00:44.441930 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:44.441963 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:44.442180 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:44.447315 (+ 5135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:44.447323 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:44.447362 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 01:00:44.447797 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:44.447812 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:44.449610 (+ 1798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:44.449618 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:44.453600 (+ 3982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:44.453620 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:44.455593 (+ 1973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:44.455599 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:44.455853 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:44.455904 (+ 51us) client_negotiation.cc:769] Sending connection context
0924 01:00:44.456028 (+ 124us) client_negotiation.cc:240] Negotiation successful
0924 01:00:44.456230 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"mutex_wait_us":27,"thread_start_us":352,"threads_started":1}
I20260924 01:00:44.457353 26129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:44.441277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49504 (local address 127.8.64.129:37765)
0924 01:00:44.446759 (+ 5482us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:44.446779 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:44.446802 (+ 23us) server_negotiation.cc:407] Connection header received
0924 01:00:44.446885 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:44.446891 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:44.447100 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:44.447321 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 01:00:44.447999 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:44.449471 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:44.453778 (+ 4307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:44.455363 (+ 1585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:44.455504 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:44.455726 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:44.456924 (+ 1198us) server_negotiation.cc:299] Negotiation successful
0924 01:00:44.457076 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5335,"thread_start_us":5210,"threads_started":1}
I20260924 01:00:44.458408 26131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:44.441909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42938 (local address 127.8.64.130:37165)
0924 01:00:44.442936 (+ 1027us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:44.442942 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:44.448485 (+ 5543us) server_negotiation.cc:407] Connection header received
0924 01:00:44.448828 (+ 343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:44.448836 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:44.449102 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:44.449307 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 01:00:44.450597 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:44.452665 (+ 2068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:44.456005 (+ 3340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:44.457161 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:44.457306 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:44.457486 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:44.457912 (+ 426us) server_negotiation.cc:299] Negotiation successful
0924 01:00:44.458068 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":865,"thread_start_us":728,"threads_started":1}
I20260924 01:00:44.458446 26130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:44.441249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.130:37165 (local address 127.0.0.1:42938)
0924 01:00:44.448364 (+ 7115us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:44.448384 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:44.448600 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:44.449644 (+ 1044us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:44.449652 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:44.449692 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 01:00:44.450330 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:44.450342 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:44.452899 (+ 2557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:44.452907 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:44.455853 (+ 2946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:44.455864 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:44.457393 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:44.457401 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:44.457693 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:44.457730 (+ 37us) client_negotiation.cc:769] Sending connection context
0924 01:00:44.457849 (+ 119us) client_negotiation.cc:240] Negotiation successful
0924 01:00:44.457981 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6990,"mutex_wait_us":710,"thread_start_us":6867,"threads_started":1}
I20260924 01:00:44.461690 26133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:44.450498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.131:37681 (local address 127.0.0.1:34824)
0924 01:00:44.453819 (+ 3321us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:44.453834 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:44.453968 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:44.454573 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:44.454577 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:44.454600 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 01:00:44.454989 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:44.454999 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:44.456767 (+ 1768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:44.456773 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:44.459199 (+ 2426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:44.459215 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:44.460967 (+ 1752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:44.460979 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:44.461144 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:44.461178 (+ 34us) client_negotiation.cc:769] Sending connection context
0924 01:00:44.461273 (+ 95us) client_negotiation.cc:240] Negotiation successful
0924 01:00:44.461388 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3115,"thread_start_us":2962,"threads_started":1}
I20260924 01:00:44.461834 26132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:44.448845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34824 (local address 127.8.64.131:37681)
0924 01:00:44.453515 (+ 4670us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:44.453527 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:44.453929 (+ 402us) server_negotiation.cc:407] Connection header received
0924 01:00:44.454131 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:44.454140 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:44.454372 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:44.454568 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 01:00:44.455147 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:44.456611 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:44.459382 (+ 2771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:44.460763 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:44.460904 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:44.461107 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:44.461335 (+ 228us) server_negotiation.cc:299] Negotiation successful
0924 01:00:44.461500 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4449,"thread_start_us":4289,"threads_started":1}
I20260924 01:00:44.461758 25784 tablet_service.cc:1467] Processing CreateTablet for tablet 9d2994be1b6e4bdb8c4c098d8f3fcc82 (DEFAULT_TABLE table=test-table [id=b30e3b2487584d24bc7e3234548cac6a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 01:00:44.462106 25918 tablet_service.cc:1467] Processing CreateTablet for tablet 9d2994be1b6e4bdb8c4c098d8f3fcc82 (DEFAULT_TABLE table=test-table [id=b30e3b2487584d24bc7e3234548cac6a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 01:00:44.463366 25784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d2994be1b6e4bdb8c4c098d8f3fcc82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:00:44.463399 25918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d2994be1b6e4bdb8c4c098d8f3fcc82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:00:44.465550 26051 tablet_service.cc:1467] Processing CreateTablet for tablet 9d2994be1b6e4bdb8c4c098d8f3fcc82 (DEFAULT_TABLE table=test-table [id=b30e3b2487584d24bc7e3234548cac6a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 01:00:44.466895 26051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d2994be1b6e4bdb8c4c098d8f3fcc82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:00:44.474929 26134 tablet_bootstrap.cc:492] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 8a37b2cb7a0f486d901c9020068772fb: Bootstrap starting.
I20260924 01:00:44.477634 26135 tablet_bootstrap.cc:492] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 52e8e0bbed514465a08aef517ae553c5: Bootstrap starting.
I20260924 01:00:44.478489 26134 tablet_bootstrap.cc:654] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 8a37b2cb7a0f486d901c9020068772fb: Neither blocks nor log segments found. Creating new log.
I20260924 01:00:44.479874 26134 log.cc:824] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 8a37b2cb7a0f486d901c9020068772fb: Log is configured to *not* fsync() on all Append() calls
I20260924 01:00:44.481541 26135 tablet_bootstrap.cc:654] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 52e8e0bbed514465a08aef517ae553c5: Neither blocks nor log segments found. Creating new log.
I20260924 01:00:44.483245 26135 log.cc:824] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 52e8e0bbed514465a08aef517ae553c5: Log is configured to *not* fsync() on all Append() calls
I20260924 01:00:44.486820 26136 tablet_bootstrap.cc:492] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 27921d144ddf4cffa3b14b3b704ba6bb: Bootstrap starting.
I20260924 01:00:44.486956 26135 tablet_bootstrap.cc:492] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 52e8e0bbed514465a08aef517ae553c5: No bootstrap required, opened a new log
I20260924 01:00:44.487238 26135 ts_tablet_manager.cc:1397] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 52e8e0bbed514465a08aef517ae553c5: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260924 01:00:44.488504 26134 tablet_bootstrap.cc:492] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 8a37b2cb7a0f486d901c9020068772fb: No bootstrap required, opened a new log
I20260924 01:00:44.488767 26134 ts_tablet_manager.cc:1397] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 8a37b2cb7a0f486d901c9020068772fb: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.009s
I20260924 01:00:44.490345 26136 tablet_bootstrap.cc:654] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 27921d144ddf4cffa3b14b3b704ba6bb: Neither blocks nor log segments found. Creating new log.
I20260924 01:00:44.491588 26136 log.cc:824] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 27921d144ddf4cffa3b14b3b704ba6bb: Log is configured to *not* fsync() on all Append() calls
I20260924 01:00:44.495142 26136 tablet_bootstrap.cc:492] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 27921d144ddf4cffa3b14b3b704ba6bb: No bootstrap required, opened a new log
I20260924 01:00:44.495414 26136 ts_tablet_manager.cc:1397] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 27921d144ddf4cffa3b14b3b704ba6bb: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.001s
I20260924 01:00:44.496757 26134 raft_consensus.cc:348] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 8a37b2cb7a0f486d901c9020068772fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52e8e0bbed514465a08aef517ae553c5" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37765 } } peers { permanent_uuid: "8a37b2cb7a0f486d901c9020068772fb" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 37165 } } peers { permanent_uuid: "27921d144ddf4cffa3b14b3b704ba6bb" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 37681 } }
I20260924 01:00:44.497148 26134 raft_consensus.cc:374] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 8a37b2cb7a0f486d901c9020068772fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 01:00:44.497275 26134 raft_consensus.cc:729] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 8a37b2cb7a0f486d901c9020068772fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a37b2cb7a0f486d901c9020068772fb, State: Initialized, Role: FOLLOWER
I20260924 01:00:44.497889 26134 consensus_queue.cc:260] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 8a37b2cb7a0f486d901c9020068772fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52e8e0bbed514465a08aef517ae553c5" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37765 } } peers { permanent_uuid: "8a37b2cb7a0f486d901c9020068772fb" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 37165 } } peers { permanent_uuid: "27921d144ddf4cffa3b14b3b704ba6bb" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 37681 } }
I20260924 01:00:44.497855 26135 raft_consensus.cc:348] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 52e8e0bbed514465a08aef517ae553c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52e8e0bbed514465a08aef517ae553c5" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37765 } } peers { permanent_uuid: "8a37b2cb7a0f486d901c9020068772fb" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 37165 } } peers { permanent_uuid: "27921d144ddf4cffa3b14b3b704ba6bb" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 37681 } }
I20260924 01:00:44.498306 26135 raft_consensus.cc:374] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 52e8e0bbed514465a08aef517ae553c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 01:00:44.498422 26135 raft_consensus.cc:729] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 52e8e0bbed514465a08aef517ae553c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52e8e0bbed514465a08aef517ae553c5, State: Initialized, Role: FOLLOWER
I20260924 01:00:44.499011 26135 consensus_queue.cc:260] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 52e8e0bbed514465a08aef517ae553c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52e8e0bbed514465a08aef517ae553c5" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37765 } } peers { permanent_uuid: "8a37b2cb7a0f486d901c9020068772fb" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 37165 } } peers { permanent_uuid: "27921d144ddf4cffa3b14b3b704ba6bb" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 37681 } }
I20260924 01:00:44.502164 26135 ts_tablet_manager.cc:1428] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 52e8e0bbed514465a08aef517ae553c5: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
I20260924 01:00:44.504798 26136 raft_consensus.cc:348] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 27921d144ddf4cffa3b14b3b704ba6bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52e8e0bbed514465a08aef517ae553c5" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37765 } } peers { permanent_uuid: "8a37b2cb7a0f486d901c9020068772fb" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 37165 } } peers { permanent_uuid: "27921d144ddf4cffa3b14b3b704ba6bb" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 37681 } }
I20260924 01:00:44.505288 26136 raft_consensus.cc:374] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 27921d144ddf4cffa3b14b3b704ba6bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 01:00:44.505398 26136 raft_consensus.cc:729] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 27921d144ddf4cffa3b14b3b704ba6bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27921d144ddf4cffa3b14b3b704ba6bb, State: Initialized, Role: FOLLOWER
I20260924 01:00:44.506093 26136 consensus_queue.cc:260] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 27921d144ddf4cffa3b14b3b704ba6bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52e8e0bbed514465a08aef517ae553c5" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37765 } } peers { permanent_uuid: "8a37b2cb7a0f486d901c9020068772fb" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 37165 } } peers { permanent_uuid: "27921d144ddf4cffa3b14b3b704ba6bb" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 37681 } }
I20260924 01:00:44.506814 25983 heartbeater.cc:503] Master 127.8.64.190:41597 was elected leader, sending a full tablet report...
I20260924 01:00:44.507642 26134 ts_tablet_manager.cc:1428] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 8a37b2cb7a0f486d901c9020068772fb: Time spent starting tablet: real 0.019s user 0.014s sys 0.005s
I20260924 01:00:44.508716 26116 heartbeater.cc:503] Master 127.8.64.190:41597 was elected leader, sending a full tablet report...
I20260924 01:00:44.509315 26136 ts_tablet_manager.cc:1428] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 27921d144ddf4cffa3b14b3b704ba6bb: Time spent starting tablet: real 0.014s user 0.006s sys 0.007s
W20260924 01:00:44.588631 25850 tablet.cc:2369] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 52e8e0bbed514465a08aef517ae553c5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 01:00:44.604274 25984 tablet.cc:2369] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 8a37b2cb7a0f486d901c9020068772fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 01:00:44.615063 26117 tablet.cc:2369] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 27921d144ddf4cffa3b14b3b704ba6bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 01:00:44.773047 26142 raft_consensus.cc:482] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 27921d144ddf4cffa3b14b3b704ba6bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 01:00:44.773401 26142 raft_consensus.cc:504] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 27921d144ddf4cffa3b14b3b704ba6bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52e8e0bbed514465a08aef517ae553c5" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37765 } } peers { permanent_uuid: "8a37b2cb7a0f486d901c9020068772fb" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 37165 } } peers { permanent_uuid: "27921d144ddf4cffa3b14b3b704ba6bb" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 37681 } }
I20260924 01:00:44.775166 26142 leader_election.cc:288] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 27921d144ddf4cffa3b14b3b704ba6bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52e8e0bbed514465a08aef517ae553c5 (127.8.64.129:37765), 8a37b2cb7a0f486d901c9020068772fb (127.8.64.130:37165)
I20260924 01:00:44.781047 26131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:44.775986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:47559 (local address 127.8.64.130:37165)
0924 01:00:44.776168 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:44.776172 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:44.776187 (+ 15us) server_negotiation.cc:407] Connection header received
0924 01:00:44.776249 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:44.776253 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:44.776307 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:44.776378 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 01:00:44.776919 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:44.777640 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:44.779371 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:44.780140 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:44.780322 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:44.780400 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:44.780647 (+ 247us) server_negotiation.cc:299] Negotiation successful
0924 01:00:44.780774 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260924 01:00:44.781052 26118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:44.775526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.130:37165 (local address 127.8.64.131:47559)
0924 01:00:44.775749 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:44.775771 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:44.775910 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:44.776434 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:44.776444 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:44.776490 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 01:00:44.776769 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:44.776779 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:44.777775 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:44.777783 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:44.779189 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:44.779203 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:44.780296 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:44.780303 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:44.780488 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:44.780512 (+ 24us) client_negotiation.cc:769] Sending connection context
0924 01:00:44.780620 (+ 108us) client_negotiation.cc:240] Negotiation successful
0924 01:00:44.780748 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59,"spinlock_wait_cycles":35712}
I20260924 01:00:44.781513 26129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:44.775587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:33791 (local address 127.8.64.129:37765)
0924 01:00:44.775803 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:44.775811 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:44.776559 (+ 748us) server_negotiation.cc:407] Connection header received
0924 01:00:44.776733 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:44.776737 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:44.776799 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:44.776901 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 01:00:44.777421 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:44.778116 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:44.779848 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:44.780623 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:44.780826 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:44.780898 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:44.781148 (+ 250us) server_negotiation.cc:299] Negotiation successful
0924 01:00:44.781272 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260924 01:00:44.781599 26143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:44.775613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:37765 (local address 127.8.64.131:33791)
0924 01:00:44.776450 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:44.776466 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:44.776601 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:44.776922 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:44.776927 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:44.776952 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 01:00:44.777257 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:44.777266 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:44.778280 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:44.778286 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:44.779697 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:44.779709 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:44.780823 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:44.780830 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:44.780994 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:44.781018 (+ 24us) client_negotiation.cc:769] Sending connection context
0924 01:00:44.781123 (+ 105us) client_negotiation.cc:240] Negotiation successful
0924 01:00:44.781241 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":695,"thread_start_us":491,"threads_started":1}
I20260924 01:00:44.782168 25938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d2994be1b6e4bdb8c4c098d8f3fcc82" candidate_uuid: "27921d144ddf4cffa3b14b3b704ba6bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a37b2cb7a0f486d901c9020068772fb" is_pre_election: true
I20260924 01:00:44.782594 25804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d2994be1b6e4bdb8c4c098d8f3fcc82" candidate_uuid: "27921d144ddf4cffa3b14b3b704ba6bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52e8e0bbed514465a08aef517ae553c5" is_pre_election: true
I20260924 01:00:44.782711 25938 raft_consensus.cc:2454] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 8a37b2cb7a0f486d901c9020068772fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27921d144ddf4cffa3b14b3b704ba6bb in term 0.
I20260924 01:00:44.783051 25804 raft_consensus.cc:2454] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 52e8e0bbed514465a08aef517ae553c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27921d144ddf4cffa3b14b3b704ba6bb in term 0.
I20260924 01:00:44.783397 26006 leader_election.cc:302] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 27921d144ddf4cffa3b14b3b704ba6bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27921d144ddf4cffa3b14b3b704ba6bb, 8a37b2cb7a0f486d901c9020068772fb; no voters:
I20260924 01:00:44.783795 26142 raft_consensus.cc:2781] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 27921d144ddf4cffa3b14b3b704ba6bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 01:00:44.783950 26142 raft_consensus.cc:482] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 27921d144ddf4cffa3b14b3b704ba6bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 01:00:44.784071 26142 raft_consensus.cc:3037] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 27921d144ddf4cffa3b14b3b704ba6bb [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:00:44.786222 26142 raft_consensus.cc:504] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 27921d144ddf4cffa3b14b3b704ba6bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52e8e0bbed514465a08aef517ae553c5" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37765 } } peers { permanent_uuid: "8a37b2cb7a0f486d901c9020068772fb" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 37165 } } peers { permanent_uuid: "27921d144ddf4cffa3b14b3b704ba6bb" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 37681 } }
I20260924 01:00:44.786849 26142 leader_election.cc:288] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 27921d144ddf4cffa3b14b3b704ba6bb [CANDIDATE]: Term 1 election: Requested vote from peers 52e8e0bbed514465a08aef517ae553c5 (127.8.64.129:37765), 8a37b2cb7a0f486d901c9020068772fb (127.8.64.130:37165)
I20260924 01:00:44.787300 25804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d2994be1b6e4bdb8c4c098d8f3fcc82" candidate_uuid: "27921d144ddf4cffa3b14b3b704ba6bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52e8e0bbed514465a08aef517ae553c5"
I20260924 01:00:44.787369 25938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d2994be1b6e4bdb8c4c098d8f3fcc82" candidate_uuid: "27921d144ddf4cffa3b14b3b704ba6bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a37b2cb7a0f486d901c9020068772fb"
I20260924 01:00:44.787493 25804 raft_consensus.cc:3037] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 52e8e0bbed514465a08aef517ae553c5 [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:00:44.787642 25938 raft_consensus.cc:3037] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 8a37b2cb7a0f486d901c9020068772fb [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:00:44.789726 25804 raft_consensus.cc:2454] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 52e8e0bbed514465a08aef517ae553c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27921d144ddf4cffa3b14b3b704ba6bb in term 1.
I20260924 01:00:44.789780 25938 raft_consensus.cc:2454] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 8a37b2cb7a0f486d901c9020068772fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27921d144ddf4cffa3b14b3b704ba6bb in term 1.
I20260924 01:00:44.790158 26007 leader_election.cc:302] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 27921d144ddf4cffa3b14b3b704ba6bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27921d144ddf4cffa3b14b3b704ba6bb, 52e8e0bbed514465a08aef517ae553c5; no voters:
I20260924 01:00:44.790513 26142 raft_consensus.cc:2781] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 27921d144ddf4cffa3b14b3b704ba6bb [term 1 FOLLOWER]: Leader election won for term 1
I20260924 01:00:44.791028 26142 raft_consensus.cc:686] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 27921d144ddf4cffa3b14b3b704ba6bb [term 1 LEADER]: Becoming Leader. State: Replica: 27921d144ddf4cffa3b14b3b704ba6bb, State: Running, Role: LEADER
I20260924 01:00:44.791497 26142 consensus_queue.cc:237] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 27921d144ddf4cffa3b14b3b704ba6bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52e8e0bbed514465a08aef517ae553c5" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37765 } } peers { permanent_uuid: "8a37b2cb7a0f486d901c9020068772fb" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 37165 } } peers { permanent_uuid: "27921d144ddf4cffa3b14b3b704ba6bb" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 37681 } }
I20260924 01:00:44.795938 25660 catalog_manager.cc:5495] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 27921d144ddf4cffa3b14b3b704ba6bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 27921d144ddf4cffa3b14b3b704ba6bb (127.8.64.131). New cstate: current_term: 1 leader_uuid: "27921d144ddf4cffa3b14b3b704ba6bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52e8e0bbed514465a08aef517ae553c5" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37765 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a37b2cb7a0f486d901c9020068772fb" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 37165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27921d144ddf4cffa3b14b3b704ba6bb" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 37681 } health_report { overall_health: HEALTHY } } }
I20260924 01:00:44.819031 26132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:44.813568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34836 (local address 127.8.64.131:37681)
0924 01:00:44.813735 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:44.813741 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:44.813912 (+ 171us) server_negotiation.cc:407] Connection header received
0924 01:00:44.814172 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:44.814176 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:44.814252 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:44.814468 (+ 216us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 01:00:44.814817 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:44.815487 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:44.816938 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:44.817318 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:44.817408 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:44.817871 (+ 463us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 01:00:44.818441 (+ 570us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 01:00:44.818569 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:44.818676 (+ 107us) server_negotiation.cc:299] Negotiation successful
0924 01:00:44.818737 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260924 01:00:44.826655 25804 raft_consensus.cc:1261] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 52e8e0bbed514465a08aef517ae553c5 [term 1 FOLLOWER]: Refusing update from remote peer 27921d144ddf4cffa3b14b3b704ba6bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 01:00:44.826670 25938 raft_consensus.cc:1261] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 8a37b2cb7a0f486d901c9020068772fb [term 1 FOLLOWER]: Refusing update from remote peer 27921d144ddf4cffa3b14b3b704ba6bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 01:00:44.827636 26144 consensus_queue.cc:1035] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 27921d144ddf4cffa3b14b3b704ba6bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "52e8e0bbed514465a08aef517ae553c5" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 37765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 01:00:44.828051 26142 consensus_queue.cc:1035] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 27921d144ddf4cffa3b14b3b704ba6bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a37b2cb7a0f486d901c9020068772fb" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 37165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 01:00:44.854574 25660 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52972:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 01:00:44.854838 25660 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52972:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 01:00:44.858283 25660 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f80d379c1ee84cc09ba87b5ea0615bb4: Sending DeleteTablet for 3 replicas of tablet 9d2994be1b6e4bdb8c4c098d8f3fcc82
I20260924 01:00:44.859460 25918 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d2994be1b6e4bdb8c4c098d8f3fcc82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 01:00:44 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:42938
I20260924 01:00:44.859625 26051 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d2994be1b6e4bdb8c4c098d8f3fcc82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 01:00:44 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:34824
I20260924 01:00:44.860517 25784 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d2994be1b6e4bdb8c4c098d8f3fcc82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 01:00:44 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:49504
I20260924 01:00:44.864650 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 25721
I20260924 01:00:44.874496 26158 tablet_replica.cc:330] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 27921d144ddf4cffa3b14b3b704ba6bb: stopping tablet replica
I20260924 01:00:44.875100 26158 raft_consensus.cc:2229] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 27921d144ddf4cffa3b14b3b704ba6bb [term 1 LEADER]: Raft consensus shutting down.
I20260924 01:00:44.875841 26158 raft_consensus.cc:2258] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 27921d144ddf4cffa3b14b3b704ba6bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 01:00:44.878221 26158 ts_tablet_manager.cc:1905] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 27921d144ddf4cffa3b14b3b704ba6bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 01:00:44.879606 26159 tablet_replica.cc:330] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 52e8e0bbed514465a08aef517ae553c5: stopping tablet replica
I20260924 01:00:44.880475 26159 raft_consensus.cc:2229] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 52e8e0bbed514465a08aef517ae553c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 01:00:44.881076 26157 tablet_replica.cc:330] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 8a37b2cb7a0f486d901c9020068772fb: stopping tablet replica
I20260924 01:00:44.881631 26159 raft_consensus.cc:2258] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 52e8e0bbed514465a08aef517ae553c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 01:00:44.882082 26157 raft_consensus.cc:2229] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 8a37b2cb7a0f486d901c9020068772fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 01:00:44.884347 26159 ts_tablet_manager.cc:1905] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 52e8e0bbed514465a08aef517ae553c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 01:00:44.884851 26157 raft_consensus.cc:2258] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 8a37b2cb7a0f486d901c9020068772fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 01:00:44.885854 26158 ts_tablet_manager.cc:1918] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 27921d144ddf4cffa3b14b3b704ba6bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 01:00:44.886026 26158 log.cc:1196] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 27921d144ddf4cffa3b14b3b704ba6bb: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-2/wal/wals/9d2994be1b6e4bdb8c4c098d8f3fcc82
I20260924 01:00:44.886638 26158 ts_tablet_manager.cc:1939] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 27921d144ddf4cffa3b14b3b704ba6bb: Deleting consensus metadata
I20260924 01:00:44.887351 26157 ts_tablet_manager.cc:1905] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 8a37b2cb7a0f486d901c9020068772fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 01:00:44.888569 25646 catalog_manager.cc:4841] TS 27921d144ddf4cffa3b14b3b704ba6bb (127.8.64.131:37681): tablet 9d2994be1b6e4bdb8c4c098d8f3fcc82 (table test-table [id=b30e3b2487584d24bc7e3234548cac6a]) successfully deleted
Sep 24 01:00:44 dist-test-slave-mmhs krb5kdc[25615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.129@KRBTEST.COM
I20260924 01:00:44.895843 26157 ts_tablet_manager.cc:1918] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 8a37b2cb7a0f486d901c9020068772fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 01:00:44.896039 26157 log.cc:1196] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 8a37b2cb7a0f486d901c9020068772fb: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-1/wal/wals/9d2994be1b6e4bdb8c4c098d8f3fcc82
I20260924 01:00:44.896545 26159 ts_tablet_manager.cc:1918] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 52e8e0bbed514465a08aef517ae553c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 01:00:44.896607 26157 ts_tablet_manager.cc:1939] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 8a37b2cb7a0f486d901c9020068772fb: Deleting consensus metadata
I20260924 01:00:44.896726 26159 log.cc:1196] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 52e8e0bbed514465a08aef517ae553c5: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790211343099518-8450-0/minicluster-data/ts-0/wal/wals/9d2994be1b6e4bdb8c4c098d8f3fcc82
I20260924 01:00:44.897305 26159 ts_tablet_manager.cc:1939] T 9d2994be1b6e4bdb8c4c098d8f3fcc82 P 52e8e0bbed514465a08aef517ae553c5: Deleting consensus metadata
I20260924 01:00:44.898500 25646 catalog_manager.cc:4841] TS 8a37b2cb7a0f486d901c9020068772fb (127.8.64.130:37165): tablet 9d2994be1b6e4bdb8c4c098d8f3fcc82 (table test-table [id=b30e3b2487584d24bc7e3234548cac6a]) successfully deleted
I20260924 01:00:44.899197 26129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:44.866279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49514 (local address 127.8.64.129:37765)
0924 01:00:44.866611 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:44.866620 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:44.878932 (+ 12312us) server_negotiation.cc:407] Connection header received
0924 01:00:44.879140 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:44.879150 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:44.879262 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:44.879381 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:44.881372 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:44.882488 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:44.884439 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:44.885083 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:44.885181 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:44.890428 (+ 5247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:44.890445 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:44.890462 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:00:44.890510 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:44.894585 (+ 4075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:44.895566 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:44.895576 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:44.895580 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:44.895740 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:44.896415 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:44.896419 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:44.896422 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:44.897237 (+ 815us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:44.897621 (+ 384us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:44.898535 (+ 914us) server_negotiation.cc:299] Negotiation successful
0924 01:00:44.898761 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162}
I20260924 01:00:44.899747 25647 catalog_manager.cc:4841] TS 52e8e0bbed514465a08aef517ae553c5 (127.8.64.129:37765): tablet 9d2994be1b6e4bdb8c4c098d8f3fcc82 (table test-table [id=b30e3b2487584d24bc7e3234548cac6a]) successfully deleted
I20260924 01:00:44.899884 25844 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 01:00:44.990778 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 25721
I20260924 01:00:45.014068 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 25855
Sep 24 01:00:45 dist-test-slave-mmhs krb5kdc[25615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.130@KRBTEST.COM
I20260924 01:00:45.031174 26131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:45.014966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42954 (local address 127.8.64.130:37165)
0924 01:00:45.015192 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:45.015198 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:45.015221 (+ 23us) server_negotiation.cc:407] Connection header received
0924 01:00:45.015420 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:45.015427 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:45.015514 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:45.015630 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:45.016826 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:45.017880 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:45.019208 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:45.019836 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:45.019968 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:45.024314 (+ 4346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:45.024334 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:45.024355 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:00:45.024403 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:45.028283 (+ 3880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:45.028945 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:45.028957 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:45.028961 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:45.029123 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:45.029466 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:45.029471 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:45.029475 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:45.029962 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:45.030248 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:45.030555 (+ 307us) server_negotiation.cc:299] Negotiation successful
0924 01:00:45.030778 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260924 01:00:45.031872 25978 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 01:00:45.123313 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 25855
I20260924 01:00:45.147776 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 25988
Sep 24 01:00:45 dist-test-slave-mmhs krb5kdc[25615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.131@KRBTEST.COM
I20260924 01:00:45.167132 26132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:45.148774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34844 (local address 127.8.64.131:37681)
0924 01:00:45.148954 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:45.148958 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:45.148977 (+ 19us) server_negotiation.cc:407] Connection header received
0924 01:00:45.149142 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:45.149146 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:45.149216 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:45.149304 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:45.151225 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:45.152212 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:45.154313 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:45.154838 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:45.154970 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:45.160409 (+ 5439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:45.160423 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:45.160438 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:00:45.160483 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:45.163808 (+ 3325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:45.164591 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:45.164599 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:45.164601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:45.164767 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:45.165241 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:45.165248 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:45.165252 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:45.165639 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:45.166010 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:45.166338 (+ 328us) server_negotiation.cc:299] Negotiation successful
0924 01:00:45.166588 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260924 01:00:45.167899 26111 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 01:00:45.294648 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 25988
I20260924 01:00:45.314411 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 25628
I20260924 01:00:45.315529 25690 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 01:00:45.424356 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 25628
2026-09-24T01:00:45Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4547 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-24T01:00:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T01:00:45Z Disabled control of system clock
I20260924 01:00:45.504817 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:45025
--webserver_interface=127.8.64.190
--webserver_port=0
--builtin_ntp_servers=127.8.64.148:34123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:45025
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 01:00:45.834267 26168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:00:45.834578 26168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:00:45.834661 26168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:00:45.841939 26168 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 01:00:45.842051 26168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:00:45.842089 26168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:00:45.842139 26168 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 01:00:45.842185 26168 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 01:00:45.851460 26168 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:34123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:45025
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:45025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:00:45.852679 26168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:00:45.854592 26168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:00:45.866173 26173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:45.866237 26174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:45.866217 26176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:00:45.867101 26168 server_base.cc:1034] running on GCE node
I20260924 01:00:45.867877 26168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:00:45.869189 26168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:00:45.870398 26168 hybrid_clock.cc:648] HybridClock initialized: now 1790211645870351 us; error 61 us; skew 500 ppm
I20260924 01:00:45.870746 26168 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 01:00:45.873088 26168 webserver.cc:460] Webserver started at http://127.8.64.190:39337/ using document root <none> and password file <none>
I20260924 01:00:45.873754 26168 fs_manager.cc:359] Metadata directory not provided
I20260924 01:00:45.873888 26168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:00:45.874212 26168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:00:45.877075 26168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "0e04792fb57c409cb3ffc914ede22e92"
format_stamp: "Formatted at 2026-09-24 01:00:45 on dist-test-slave-mmhs"
server_key: "6e80949f421e0cbf2beb60447d30c4a1"
server_key_iv: "c253663d0181a49b0d6c5757346467e6"
server_key_version: "encryptionkey@0"
I20260924 01:00:45.877877 26168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "0e04792fb57c409cb3ffc914ede22e92"
format_stamp: "Formatted at 2026-09-24 01:00:45 on dist-test-slave-mmhs"
server_key: "6e80949f421e0cbf2beb60447d30c4a1"
server_key_iv: "c253663d0181a49b0d6c5757346467e6"
server_key_version: "encryptionkey@0"
I20260924 01:00:45.882954 26168 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260924 01:00:45.886404 26182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:45.887667 26168 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260924 01:00:45.887863 26168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "0e04792fb57c409cb3ffc914ede22e92"
format_stamp: "Formatted at 2026-09-24 01:00:45 on dist-test-slave-mmhs"
server_key: "6e80949f421e0cbf2beb60447d30c4a1"
server_key_iv: "c253663d0181a49b0d6c5757346467e6"
server_key_version: "encryptionkey@0"
I20260924 01:00:45.888088 26168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:00:45.916673 26168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:00:45.917547 26168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:00:45.917852 26168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:00:45.936883 26168 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.190:45025
I20260924 01:00:45.936903 26233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.190:45025 every 8 connection(s)
I20260924 01:00:45.938235 26168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 01:00:45.941941 26234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:00:45.948611 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 26168
I20260924 01:00:45.948822 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 01:00:45.949278 8450 external_mini_cluster.cc:1428] Setting key 44aabeb56834269501c14a6e571aee8b
I20260924 01:00:45.952560 26234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e04792fb57c409cb3ffc914ede22e92: Bootstrap starting.
I20260924 01:00:45.955782 26234 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e04792fb57c409cb3ffc914ede22e92: Neither blocks nor log segments found. Creating new log.
I20260924 01:00:45.956835 26234 log.cc:824] T 00000000000000000000000000000000 P 0e04792fb57c409cb3ffc914ede22e92: Log is configured to *not* fsync() on all Append() calls
I20260924 01:00:45.959542 26234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e04792fb57c409cb3ffc914ede22e92: No bootstrap required, opened a new log
I20260924 01:00:45.962551 26237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:45.951117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46958 (local address 127.8.64.190:45025)
0924 01:00:45.952050 (+ 933us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:45.952060 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:45.952101 (+ 41us) server_negotiation.cc:407] Connection header received
0924 01:00:45.953972 (+ 1871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:45.953998 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:45.954513 (+ 515us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:45.954893 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:45.955441 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:45.956819 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:45.958193 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:45.958915 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:45.959112 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:45.960289 (+ 1177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:45.960296 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:45.960310 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:00:45.960809 (+ 499us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:45.960883 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:00:45.960901 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:45.961041 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:45.961251 (+ 210us) server_negotiation.cc:299] Negotiation successful
0924 01:00:45.961544 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":320,"threads_started":1}
I20260924 01:00:45.967705 26234 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e04792fb57c409cb3ffc914ede22e92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e04792fb57c409cb3ffc914ede22e92" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45025 } }
I20260924 01:00:45.968061 26234 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0e04792fb57c409cb3ffc914ede22e92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 01:00:45.968174 26234 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e04792fb57c409cb3ffc914ede22e92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e04792fb57c409cb3ffc914ede22e92, State: Initialized, Role: FOLLOWER
I20260924 01:00:45.968788 26234 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e04792fb57c409cb3ffc914ede22e92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e04792fb57c409cb3ffc914ede22e92" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45025 } }
I20260924 01:00:45.969050 26234 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e04792fb57c409cb3ffc914ede22e92 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 01:00:45.969198 26234 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e04792fb57c409cb3ffc914ede22e92 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 01:00:45.969455 26234 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e04792fb57c409cb3ffc914ede22e92 [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:00:45.971745 26234 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e04792fb57c409cb3ffc914ede22e92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e04792fb57c409cb3ffc914ede22e92" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45025 } }
I20260924 01:00:45.972183 26234 leader_election.cc:302] T 00000000000000000000000000000000 P 0e04792fb57c409cb3ffc914ede22e92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e04792fb57c409cb3ffc914ede22e92; no voters:
I20260924 01:00:45.972829 26234 leader_election.cc:288] T 00000000000000000000000000000000 P 0e04792fb57c409cb3ffc914ede22e92 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 01:00:45.972986 26239 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e04792fb57c409cb3ffc914ede22e92 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 01:00:45.973718 26239 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e04792fb57c409cb3ffc914ede22e92 [term 1 LEADER]: Becoming Leader. State: Replica: 0e04792fb57c409cb3ffc914ede22e92, State: Running, Role: LEADER
I20260924 01:00:45.974336 26239 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e04792fb57c409cb3ffc914ede22e92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e04792fb57c409cb3ffc914ede22e92" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45025 } }
I20260924 01:00:45.974972 26234 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e04792fb57c409cb3ffc914ede22e92 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 01:00:45.976869 26240 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e04792fb57c409cb3ffc914ede22e92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e04792fb57c409cb3ffc914ede22e92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e04792fb57c409cb3ffc914ede22e92" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45025 } } }
I20260924 01:00:45.977276 26240 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e04792fb57c409cb3ffc914ede22e92 [sys.catalog]: This master's current role is: LEADER
I20260924 01:00:45.977254 26241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e04792fb57c409cb3ffc914ede22e92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e04792fb57c409cb3ffc914ede22e92. Latest consensus state: current_term: 1 leader_uuid: "0e04792fb57c409cb3ffc914ede22e92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e04792fb57c409cb3ffc914ede22e92" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45025 } } }
I20260924 01:00:45.977555 26241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e04792fb57c409cb3ffc914ede22e92 [sys.catalog]: This master's current role is: LEADER
I20260924 01:00:45.979518 26244 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 01:00:45.984988 26244 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 01:00:45.994065 26244 catalog_manager.cc:1347] Generated new cluster ID: ed7004e541204d4abf39f8adf8d8e56b
I20260924 01:00:45.994199 26244 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 01:00:46.009202 26244 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 01:00:46.010517 26244 catalog_manager.cc:1504] Loading token signing keys...
I20260924 01:00:46.021204 26244 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0e04792fb57c409cb3ffc914ede22e92: Generated new TSK 0
I20260924 01:00:46.022020 26244 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 01:00:46.031322 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.129:0
--local_ip_for_outbound_sockets=127.8.64.129
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:45025
--builtin_ntp_servers=127.8.64.148:34123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 01:00:46.373571 26258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:00:46.373972 26258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:00:46.374161 26258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:00:46.385933 26258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:00:46.386040 26258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:00:46.386250 26258 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.129
I20260924 01:00:46.400534 26258 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:34123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:45025
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.129
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:00:46.402480 26258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:00:46.404944 26258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:00:46.419667 26263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:46.420491 26264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:46.420974 26266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:00:46.422379 26258 server_base.cc:1034] running on GCE node
I20260924 01:00:46.423193 26258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:00:46.424546 26258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:00:46.425915 26258 hybrid_clock.cc:648] HybridClock initialized: now 1790211646425792 us; error 84 us; skew 500 ppm
I20260924 01:00:46.426415 26258 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 01:00:46.429481 26258 webserver.cc:460] Webserver started at http://127.8.64.129:43975/ using document root <none> and password file <none>
I20260924 01:00:46.430153 26258 fs_manager.cc:359] Metadata directory not provided
I20260924 01:00:46.430277 26258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:00:46.430786 26258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:00:46.433851 26258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-0/data/instance:
uuid: "07fb13b3dddd4fa38aa66a2fe9e190c1"
format_stamp: "Formatted at 2026-09-24 01:00:46 on dist-test-slave-mmhs"
server_key: "f1f2338fa9acfb15ac39293dcb4e08b9"
server_key_iv: "ff209c78cab958dbe99e82605feebf29"
server_key_version: "encryptionkey@0"
I20260924 01:00:46.434695 26258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance:
uuid: "07fb13b3dddd4fa38aa66a2fe9e190c1"
format_stamp: "Formatted at 2026-09-24 01:00:46 on dist-test-slave-mmhs"
server_key: "f1f2338fa9acfb15ac39293dcb4e08b9"
server_key_iv: "ff209c78cab958dbe99e82605feebf29"
server_key_version: "encryptionkey@0"
I20260924 01:00:46.439662 26258 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260924 01:00:46.444000 26272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:46.445230 26258 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260924 01:00:46.445413 26258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-0/wal
uuid: "07fb13b3dddd4fa38aa66a2fe9e190c1"
format_stamp: "Formatted at 2026-09-24 01:00:46 on dist-test-slave-mmhs"
server_key: "f1f2338fa9acfb15ac39293dcb4e08b9"
server_key_iv: "ff209c78cab958dbe99e82605feebf29"
server_key_version: "encryptionkey@0"
I20260924 01:00:46.445636 26258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:00:46.465107 26258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:00:46.465904 26258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:00:46.466218 26258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:00:46.467178 26258 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 01:00:46.468690 26258 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 01:00:46.468816 26258 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:46.468936 26258 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 01:00:46.469024 26258 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:46.502120 26258 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.129:35195
I20260924 01:00:46.502149 26384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.129:35195 every 8 connection(s)
I20260924 01:00:46.503427 26258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
I20260924 01:00:46.509483 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 26258
I20260924 01:00:46.509671 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance
I20260924 01:00:46.510231 8450 external_mini_cluster.cc:1428] Setting key dbd819a58386d13f86130317e1642293
I20260924 01:00:46.513473 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.130:0
--local_ip_for_outbound_sockets=127.8.64.130
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:45025
--builtin_ntp_servers=127.8.64.148:34123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260924 01:00:46.517377 26387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:46.505911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:56495 (local address 127.8.64.190:45025)
0924 01:00:46.506415 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:46.506419 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:46.507456 (+ 1037us) server_negotiation.cc:407] Connection header received
0924 01:00:46.509262 (+ 1806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:46.509266 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:46.509337 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:46.509435 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:46.510876 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:46.511982 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:46.514461 (+ 2479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:46.514858 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:46.514959 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:46.516629 (+ 1670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:46.516633 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:46.516637 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:00:46.516654 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:46.516703 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:00:46.516715 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:46.516783 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:46.517049 (+ 266us) server_negotiation.cc:299] Negotiation successful
0924 01:00:46.517134 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":247,"threads_started":1}
I20260924 01:00:46.518857 26388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:46.506438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45025 (local address 127.8.64.129:56495)
0924 01:00:46.507300 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:46.507344 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:46.508953 (+ 1609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:46.509717 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:46.509729 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:46.509845 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0924 01:00:46.510669 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:46.510694 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:46.512160 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:46.512165 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:46.514280 (+ 2115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:46.514327 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:46.515090 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:46.515100 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:46.515441 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:46.516245 (+ 804us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 01:00:46.516250 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 01:00:46.516306 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 01:00:46.516425 (+ 119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 01:00:46.516811 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 01:00:46.516823 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 01:00:46.516834 (+ 11us) client_negotiation.cc:769] Sending connection context
0924 01:00:46.517023 (+ 189us) client_negotiation.cc:240] Negotiation successful
0924 01:00:46.517456 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":273,"threads_started":1}
I20260924 01:00:46.520403 26385 heartbeater.cc:347] Connected to a master server at 127.8.64.190:45025
I20260924 01:00:46.520778 26385 heartbeater.cc:464] Registering TS with master...
I20260924 01:00:46.521643 26385 heartbeater.cc:511] Master 127.8.64.190:45025 requested a full tablet report, sending...
I20260924 01:00:46.523937 26199 ts_manager.cc:194] Registered new tserver with Master: 07fb13b3dddd4fa38aa66a2fe9e190c1 (127.8.64.129:35195)
I20260924 01:00:46.526136 26199 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.64.129:56495
W20260924 01:00:46.848615 26389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:00:46.848893 26389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:00:46.849040 26389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:00:46.856559 26389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:00:46.856658 26389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:00:46.856765 26389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.130
I20260924 01:00:46.865756 26389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:34123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:45025
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.130
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:00:46.866989 26389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:00:46.868835 26389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:00:46.881875 26395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:46.881945 26394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:46.881978 26397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:00:46.888199 26389 server_base.cc:1034] running on GCE node
I20260924 01:00:46.888896 26389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:00:46.890008 26389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:00:46.891263 26389 hybrid_clock.cc:648] HybridClock initialized: now 1790211646891199 us; error 101 us; skew 500 ppm
I20260924 01:00:46.891618 26389 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 01:00:46.894134 26389 webserver.cc:460] Webserver started at http://127.8.64.130:38265/ using document root <none> and password file <none>
I20260924 01:00:46.894938 26389 fs_manager.cc:359] Metadata directory not provided
I20260924 01:00:46.895094 26389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:00:46.895444 26389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:00:46.898602 26389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-1/data/instance:
uuid: "f142e4eefd1e40f49ecda7adf57bb6e3"
format_stamp: "Formatted at 2026-09-24 01:00:46 on dist-test-slave-mmhs"
server_key: "163122a8d93a29cae75b0670eb35f0e1"
server_key_iv: "e11507c7467673d8781b31e92a973c46"
server_key_version: "encryptionkey@0"
I20260924 01:00:46.899461 26389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance:
uuid: "f142e4eefd1e40f49ecda7adf57bb6e3"
format_stamp: "Formatted at 2026-09-24 01:00:46 on dist-test-slave-mmhs"
server_key: "163122a8d93a29cae75b0670eb35f0e1"
server_key_iv: "e11507c7467673d8781b31e92a973c46"
server_key_version: "encryptionkey@0"
I20260924 01:00:46.904843 26389 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260924 01:00:46.908618 26403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:46.909950 26389 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260924 01:00:46.910157 26389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-1/wal
uuid: "f142e4eefd1e40f49ecda7adf57bb6e3"
format_stamp: "Formatted at 2026-09-24 01:00:46 on dist-test-slave-mmhs"
server_key: "163122a8d93a29cae75b0670eb35f0e1"
server_key_iv: "e11507c7467673d8781b31e92a973c46"
server_key_version: "encryptionkey@0"
I20260924 01:00:46.910480 26389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:00:46.934093 26389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:00:46.935114 26389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:00:46.935464 26389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:00:46.936707 26389 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 01:00:46.938364 26389 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 01:00:46.938494 26389 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:46.938643 26389 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 01:00:46.938740 26389 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:46.975713 26389 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.130:33807
I20260924 01:00:46.975759 26515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.130:33807 every 8 connection(s)
I20260924 01:00:46.977103 26389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
I20260924 01:00:46.983784 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 26389
I20260924 01:00:46.983992 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance
I20260924 01:00:46.984434 8450 external_mini_cluster.cc:1428] Setting key 3c1b0882f31003e0cd712c5ac11fdacb
I20260924 01:00:46.987286 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.131:0
--local_ip_for_outbound_sockets=127.8.64.131
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:45025
--builtin_ntp_servers=127.8.64.148:34123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260924 01:00:46.990517 26387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:46.979869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:46779 (local address 127.8.64.190:45025)
0924 01:00:46.980083 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:46.980087 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:46.981435 (+ 1348us) server_negotiation.cc:407] Connection header received
0924 01:00:46.983399 (+ 1964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:46.983404 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:46.983537 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:46.983668 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:46.984904 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:46.985770 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:46.987550 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:46.988004 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:46.988087 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:46.989654 (+ 1567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:46.989660 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:46.989663 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:00:46.989679 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:46.989720 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:00:46.989733 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:46.989829 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:46.990071 (+ 242us) server_negotiation.cc:299] Negotiation successful
0924 01:00:46.990172 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260924 01:00:46.991655 26518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:46.980334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45025 (local address 127.8.64.130:46779)
0924 01:00:46.981236 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:46.981276 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:46.983038 (+ 1762us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:46.983921 (+ 883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:46.983934 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:46.984068 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0924 01:00:46.984733 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:46.984753 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:46.985927 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:46.985938 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:46.987274 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:46.987286 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:46.988341 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:46.988346 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:46.988570 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:46.989310 (+ 740us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 01:00:46.989313 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0924 01:00:46.989354 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 01:00:46.989442 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 01:00:46.989828 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 01:00:46.989841 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 01:00:46.989852 (+ 11us) client_negotiation.cc:769] Sending connection context
0924 01:00:46.990057 (+ 205us) client_negotiation.cc:240] Negotiation successful
0924 01:00:46.990490 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":279,"threads_started":1}
I20260924 01:00:46.993185 26516 heartbeater.cc:347] Connected to a master server at 127.8.64.190:45025
I20260924 01:00:46.993530 26516 heartbeater.cc:464] Registering TS with master...
I20260924 01:00:46.994354 26516 heartbeater.cc:511] Master 127.8.64.190:45025 requested a full tablet report, sending...
I20260924 01:00:46.995800 26199 ts_manager.cc:194] Registered new tserver with Master: f142e4eefd1e40f49ecda7adf57bb6e3 (127.8.64.130:33807)
I20260924 01:00:46.996630 26199 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.64.130:46779
W20260924 01:00:47.316617 26519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:00:47.316947 26519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:00:47.317116 26519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:00:47.324388 26519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:00:47.324541 26519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:00:47.324725 26519 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.131
I20260924 01:00:47.333974 26519 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:34123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:45025
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.131
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:00:47.335366 26519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:00:47.337209 26519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:00:47.350425 26524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:47.350504 26527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:47.350456 26525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:00:47.351497 26519 server_base.cc:1034] running on GCE node
I20260924 01:00:47.352116 26519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:00:47.353112 26519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:00:47.354370 26519 hybrid_clock.cc:648] HybridClock initialized: now 1790211647354318 us; error 45 us; skew 500 ppm
I20260924 01:00:47.354740 26519 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 01:00:47.357067 26519 webserver.cc:460] Webserver started at http://127.8.64.131:43395/ using document root <none> and password file <none>
I20260924 01:00:47.357751 26519 fs_manager.cc:359] Metadata directory not provided
I20260924 01:00:47.357872 26519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:00:47.358206 26519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:00:47.361058 26519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-2/data/instance:
uuid: "ee1dc17edd5a4bbc8496492c174fcb8e"
format_stamp: "Formatted at 2026-09-24 01:00:47 on dist-test-slave-mmhs"
server_key: "650d144186202e88e323b886909c1744"
server_key_iv: "614912b0a5bed375a794c31afb3e96f0"
server_key_version: "encryptionkey@0"
I20260924 01:00:47.361821 26519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance:
uuid: "ee1dc17edd5a4bbc8496492c174fcb8e"
format_stamp: "Formatted at 2026-09-24 01:00:47 on dist-test-slave-mmhs"
server_key: "650d144186202e88e323b886909c1744"
server_key_iv: "614912b0a5bed375a794c31afb3e96f0"
server_key_version: "encryptionkey@0"
I20260924 01:00:47.366750 26519 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260924 01:00:47.370177 26533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:47.371567 26519 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260924 01:00:47.371740 26519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-2/wal
uuid: "ee1dc17edd5a4bbc8496492c174fcb8e"
format_stamp: "Formatted at 2026-09-24 01:00:47 on dist-test-slave-mmhs"
server_key: "650d144186202e88e323b886909c1744"
server_key_iv: "614912b0a5bed375a794c31afb3e96f0"
server_key_version: "encryptionkey@0"
I20260924 01:00:47.371953 26519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:00:47.401047 26519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:00:47.402233 26519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:00:47.402616 26519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:00:47.403764 26519 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 01:00:47.405367 26519 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 01:00:47.405499 26519 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:47.405649 26519 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 01:00:47.405741 26519 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:47.439993 26519 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.131:39923
I20260924 01:00:47.440083 26645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.131:39923 every 8 connection(s)
I20260924 01:00:47.441318 26519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
I20260924 01:00:47.443943 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 26519
I20260924 01:00:47.444132 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance
I20260924 01:00:47.444717 8450 external_mini_cluster.cc:1428] Setting key 4f273e6bac0a04a2c90992acbab63d6e
I20260924 01:00:47.455180 26387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:47.444165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:36737 (local address 127.8.64.190:45025)
0924 01:00:47.444360 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:47.444366 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:47.446329 (+ 1963us) server_negotiation.cc:407] Connection header received
0924 01:00:47.448501 (+ 2172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:47.448506 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:47.448579 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:47.448653 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:47.449807 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:47.450710 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:47.452231 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:47.452694 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:47.452801 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:47.454190 (+ 1389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:47.454195 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:47.454199 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:00:47.454217 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:47.454250 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:00:47.454262 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:47.454474 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:47.454714 (+ 240us) server_negotiation.cc:299] Negotiation successful
0924 01:00:47.454811 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260924 01:00:47.456138 26648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:47.444579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45025 (local address 127.8.64.131:36737)
0924 01:00:47.446074 (+ 1495us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:47.446163 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:47.448179 (+ 2016us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:47.448858 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:47.448866 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:47.448956 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0924 01:00:47.449620 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:47.449645 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:47.450850 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:47.450854 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:47.452082 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:47.452095 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:47.452883 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:47.452888 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:47.453086 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:47.453860 (+ 774us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 01:00:47.453862 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0924 01:00:47.453907 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 01:00:47.453985 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 01:00:47.454469 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 01:00:47.454481 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 01:00:47.454491 (+ 10us) client_negotiation.cc:769] Sending connection context
0924 01:00:47.454691 (+ 200us) client_negotiation.cc:240] Negotiation successful
0924 01:00:47.455096 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":661,"thread_start_us":389,"threads_started":1}
I20260924 01:00:47.457486 26646 heartbeater.cc:347] Connected to a master server at 127.8.64.190:45025
I20260924 01:00:47.457810 26646 heartbeater.cc:464] Registering TS with master...
I20260924 01:00:47.458630 26646 heartbeater.cc:511] Master 127.8.64.190:45025 requested a full tablet report, sending...
I20260924 01:00:47.460325 26199 ts_manager.cc:194] Registered new tserver with Master: ee1dc17edd5a4bbc8496492c174fcb8e (127.8.64.131:39923)
I20260924 01:00:47.461165 8450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 01:00:47.461408 26199 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.64.131:36737
I20260924 01:00:47.471834 26387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:47.465751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46972 (local address 127.8.64.190:45025)
0924 01:00:47.466012 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:47.466020 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:47.466419 (+ 399us) server_negotiation.cc:407] Connection header received
0924 01:00:47.466622 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:47.466626 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:47.466700 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:47.466811 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:47.467347 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:47.468047 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:47.469406 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:47.469818 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:47.469953 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:47.471111 (+ 1158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:47.471116 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:47.471119 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:00:47.471136 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:47.471166 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:00:47.471178 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:47.471274 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:47.471455 (+ 181us) server_negotiation.cc:299] Negotiation successful
0924 01:00:47.471544 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260924 01:00:47.475674 26199 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46972:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 01:00:47.482646 26199 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 01:00:47.521382 26659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:47.495107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.131:39923 (local address 127.0.0.1:42360)
0924 01:00:47.505714 (+ 10607us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:47.505790 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:47.506038 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:47.513795 (+ 7757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:47.513828 (+ 33us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:47.513879 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 01:00:47.514501 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:47.514525 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:47.516705 (+ 2180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:47.516711 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:47.519070 (+ 2359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:47.519082 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:47.520429 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:47.520435 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:47.520648 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:47.520704 (+ 56us) client_negotiation.cc:769] Sending connection context
0924 01:00:47.520849 (+ 145us) client_negotiation.cc:240] Negotiation successful
0924 01:00:47.521033 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10241,"thread_start_us":16975,"threads_started":1}
I20260924 01:00:47.522138 26661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:47.495797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42360 (local address 127.8.64.131:39923)
0924 01:00:47.512937 (+ 17140us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:47.512949 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:47.512995 (+ 46us) server_negotiation.cc:407] Connection header received
0924 01:00:47.513141 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:47.513153 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:47.513482 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:47.513749 (+ 267us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 01:00:47.514895 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:47.516493 (+ 1598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:47.519296 (+ 2803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:47.520204 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:47.520361 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:47.520665 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:47.520911 (+ 246us) server_negotiation.cc:299] Negotiation successful
0924 01:00:47.521070 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16917,"thread_start_us":16746,"threads_started":1}
I20260924 01:00:47.523268 26658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:47.495379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:35195 (local address 127.0.0.1:48642)
0924 01:00:47.511744 (+ 16365us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:47.511771 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:47.511955 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:47.515880 (+ 3925us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:47.515884 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:47.515957 (+ 73us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 01:00:47.516425 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:47.516437 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:47.518060 (+ 1623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:47.518067 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:47.520318 (+ 2251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:47.520333 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:47.522478 (+ 2145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:47.522485 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:47.522664 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:47.522695 (+ 31us) client_negotiation.cc:769] Sending connection context
0924 01:00:47.522790 (+ 95us) client_negotiation.cc:240] Negotiation successful
0924 01:00:47.522918 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16165,"mutex_wait_us":1,"thread_start_us":9785,"threads_started":1}
I20260924 01:00:47.523564 26662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:47.495914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48642 (local address 127.8.64.129:35195)
0924 01:00:47.515332 (+ 19418us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:47.515341 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:47.515367 (+ 26us) server_negotiation.cc:407] Connection header received
0924 01:00:47.515452 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:47.515460 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:47.515656 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:47.515883 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 01:00:47.516625 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:47.517894 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:47.520562 (+ 2668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:47.521973 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:47.522093 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:47.522363 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:47.522857 (+ 494us) server_negotiation.cc:299] Negotiation successful
0924 01:00:47.523024 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19211,"thread_start_us":19111,"threads_started":1}
I20260924 01:00:47.524391 26660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:47.495907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.130:33807 (local address 127.0.0.1:40452)
0924 01:00:47.514533 (+ 18626us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:47.514552 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:47.514755 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:47.515612 (+ 857us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:47.515616 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:47.515641 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 01:00:47.516057 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:47.516074 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:47.518699 (+ 2625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:47.518705 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:47.521460 (+ 2755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:47.521474 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:47.523599 (+ 2125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:47.523605 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:47.523772 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:47.523828 (+ 56us) client_negotiation.cc:769] Sending connection context
0924 01:00:47.523927 (+ 99us) client_negotiation.cc:240] Negotiation successful
0924 01:00:47.524067 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18484,"mutex_wait_us":24,"thread_start_us":18303,"threads_started":1}
I20260924 01:00:47.524667 26663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:47.496535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40452 (local address 127.8.64.130:33807)
0924 01:00:47.508171 (+ 11636us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:47.508184 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:47.514644 (+ 6460us) server_negotiation.cc:407] Connection header received
0924 01:00:47.514948 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:47.514958 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:47.515218 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:47.515381 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 01:00:47.516253 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:47.518552 (+ 2299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:47.521667 (+ 3115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:47.523084 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:47.523196 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:47.523441 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:47.523970 (+ 529us) server_negotiation.cc:299] Negotiation successful
0924 01:00:47.524171 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11441,"thread_start_us":11301,"threads_started":1}
I20260924 01:00:47.526188 26581 tablet_service.cc:1467] Processing CreateTablet for tablet f4a30dc8ea3c4a15967d6a3502bd1f7b (DEFAULT_TABLE table=test-table [id=ff1c8053f1de44ebb8e35e1f686082b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 01:00:47.527505 26581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4a30dc8ea3c4a15967d6a3502bd1f7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:00:47.527661 26451 tablet_service.cc:1467] Processing CreateTablet for tablet f4a30dc8ea3c4a15967d6a3502bd1f7b (DEFAULT_TABLE table=test-table [id=ff1c8053f1de44ebb8e35e1f686082b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 01:00:47.528708 26451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4a30dc8ea3c4a15967d6a3502bd1f7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:00:47.529985 26385 heartbeater.cc:503] Master 127.8.64.190:45025 was elected leader, sending a full tablet report...
I20260924 01:00:47.536547 26320 tablet_service.cc:1467] Processing CreateTablet for tablet f4a30dc8ea3c4a15967d6a3502bd1f7b (DEFAULT_TABLE table=test-table [id=ff1c8053f1de44ebb8e35e1f686082b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 01:00:47.537918 26320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4a30dc8ea3c4a15967d6a3502bd1f7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:00:47.540588 26664 tablet_bootstrap.cc:492] T f4a30dc8ea3c4a15967d6a3502bd1f7b P f142e4eefd1e40f49ecda7adf57bb6e3: Bootstrap starting.
I20260924 01:00:47.543026 26665 tablet_bootstrap.cc:492] T f4a30dc8ea3c4a15967d6a3502bd1f7b P ee1dc17edd5a4bbc8496492c174fcb8e: Bootstrap starting.
I20260924 01:00:47.545158 26664 tablet_bootstrap.cc:654] T f4a30dc8ea3c4a15967d6a3502bd1f7b P f142e4eefd1e40f49ecda7adf57bb6e3: Neither blocks nor log segments found. Creating new log.
I20260924 01:00:47.546165 26664 log.cc:824] T f4a30dc8ea3c4a15967d6a3502bd1f7b P f142e4eefd1e40f49ecda7adf57bb6e3: Log is configured to *not* fsync() on all Append() calls
I20260924 01:00:47.547163 26665 tablet_bootstrap.cc:654] T f4a30dc8ea3c4a15967d6a3502bd1f7b P ee1dc17edd5a4bbc8496492c174fcb8e: Neither blocks nor log segments found. Creating new log.
I20260924 01:00:47.548357 26665 log.cc:824] T f4a30dc8ea3c4a15967d6a3502bd1f7b P ee1dc17edd5a4bbc8496492c174fcb8e: Log is configured to *not* fsync() on all Append() calls
I20260924 01:00:47.548874 26664 tablet_bootstrap.cc:492] T f4a30dc8ea3c4a15967d6a3502bd1f7b P f142e4eefd1e40f49ecda7adf57bb6e3: No bootstrap required, opened a new log
I20260924 01:00:47.549193 26664 ts_tablet_manager.cc:1397] T f4a30dc8ea3c4a15967d6a3502bd1f7b P f142e4eefd1e40f49ecda7adf57bb6e3: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260924 01:00:47.551676 26665 tablet_bootstrap.cc:492] T f4a30dc8ea3c4a15967d6a3502bd1f7b P ee1dc17edd5a4bbc8496492c174fcb8e: No bootstrap required, opened a new log
I20260924 01:00:47.552328 26668 tablet_bootstrap.cc:492] T f4a30dc8ea3c4a15967d6a3502bd1f7b P 07fb13b3dddd4fa38aa66a2fe9e190c1: Bootstrap starting.
I20260924 01:00:47.552791 26665 ts_tablet_manager.cc:1397] T f4a30dc8ea3c4a15967d6a3502bd1f7b P ee1dc17edd5a4bbc8496492c174fcb8e: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260924 01:00:47.556383 26668 tablet_bootstrap.cc:654] T f4a30dc8ea3c4a15967d6a3502bd1f7b P 07fb13b3dddd4fa38aa66a2fe9e190c1: Neither blocks nor log segments found. Creating new log.
I20260924 01:00:47.557602 26668 log.cc:824] T f4a30dc8ea3c4a15967d6a3502bd1f7b P 07fb13b3dddd4fa38aa66a2fe9e190c1: Log is configured to *not* fsync() on all Append() calls
I20260924 01:00:47.558950 26664 raft_consensus.cc:348] T f4a30dc8ea3c4a15967d6a3502bd1f7b P f142e4eefd1e40f49ecda7adf57bb6e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1dc17edd5a4bbc8496492c174fcb8e" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 39923 } } peers { permanent_uuid: "07fb13b3dddd4fa38aa66a2fe9e190c1" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35195 } } peers { permanent_uuid: "f142e4eefd1e40f49ecda7adf57bb6e3" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 33807 } }
I20260924 01:00:47.559402 26664 raft_consensus.cc:374] T f4a30dc8ea3c4a15967d6a3502bd1f7b P f142e4eefd1e40f49ecda7adf57bb6e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 01:00:47.559508 26664 raft_consensus.cc:729] T f4a30dc8ea3c4a15967d6a3502bd1f7b P f142e4eefd1e40f49ecda7adf57bb6e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f142e4eefd1e40f49ecda7adf57bb6e3, State: Initialized, Role: FOLLOWER
I20260924 01:00:47.560151 26664 consensus_queue.cc:260] T f4a30dc8ea3c4a15967d6a3502bd1f7b P f142e4eefd1e40f49ecda7adf57bb6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1dc17edd5a4bbc8496492c174fcb8e" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 39923 } } peers { permanent_uuid: "07fb13b3dddd4fa38aa66a2fe9e190c1" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35195 } } peers { permanent_uuid: "f142e4eefd1e40f49ecda7adf57bb6e3" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 33807 } }
I20260924 01:00:47.560729 26668 tablet_bootstrap.cc:492] T f4a30dc8ea3c4a15967d6a3502bd1f7b P 07fb13b3dddd4fa38aa66a2fe9e190c1: No bootstrap required, opened a new log
I20260924 01:00:47.560990 26668 ts_tablet_manager.cc:1397] T f4a30dc8ea3c4a15967d6a3502bd1f7b P 07fb13b3dddd4fa38aa66a2fe9e190c1: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260924 01:00:47.561007 26665 raft_consensus.cc:348] T f4a30dc8ea3c4a15967d6a3502bd1f7b P ee1dc17edd5a4bbc8496492c174fcb8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1dc17edd5a4bbc8496492c174fcb8e" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 39923 } } peers { permanent_uuid: "07fb13b3dddd4fa38aa66a2fe9e190c1" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35195 } } peers { permanent_uuid: "f142e4eefd1e40f49ecda7adf57bb6e3" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 33807 } }
I20260924 01:00:47.561374 26516 heartbeater.cc:503] Master 127.8.64.190:45025 was elected leader, sending a full tablet report...
I20260924 01:00:47.561425 26665 raft_consensus.cc:374] T f4a30dc8ea3c4a15967d6a3502bd1f7b P ee1dc17edd5a4bbc8496492c174fcb8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 01:00:47.561532 26665 raft_consensus.cc:729] T f4a30dc8ea3c4a15967d6a3502bd1f7b P ee1dc17edd5a4bbc8496492c174fcb8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee1dc17edd5a4bbc8496492c174fcb8e, State: Initialized, Role: FOLLOWER
I20260924 01:00:47.561894 26664 ts_tablet_manager.cc:1428] T f4a30dc8ea3c4a15967d6a3502bd1f7b P f142e4eefd1e40f49ecda7adf57bb6e3: Time spent starting tablet: real 0.013s user 0.007s sys 0.006s
I20260924 01:00:47.562124 26665 consensus_queue.cc:260] T f4a30dc8ea3c4a15967d6a3502bd1f7b P ee1dc17edd5a4bbc8496492c174fcb8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1dc17edd5a4bbc8496492c174fcb8e" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 39923 } } peers { permanent_uuid: "07fb13b3dddd4fa38aa66a2fe9e190c1" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35195 } } peers { permanent_uuid: "f142e4eefd1e40f49ecda7adf57bb6e3" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 33807 } }
I20260924 01:00:47.563246 26646 heartbeater.cc:503] Master 127.8.64.190:45025 was elected leader, sending a full tablet report...
I20260924 01:00:47.564030 26665 ts_tablet_manager.cc:1428] T f4a30dc8ea3c4a15967d6a3502bd1f7b P ee1dc17edd5a4bbc8496492c174fcb8e: Time spent starting tablet: real 0.011s user 0.003s sys 0.008s
I20260924 01:00:47.571005 26668 raft_consensus.cc:348] T f4a30dc8ea3c4a15967d6a3502bd1f7b P 07fb13b3dddd4fa38aa66a2fe9e190c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1dc17edd5a4bbc8496492c174fcb8e" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 39923 } } peers { permanent_uuid: "07fb13b3dddd4fa38aa66a2fe9e190c1" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35195 } } peers { permanent_uuid: "f142e4eefd1e40f49ecda7adf57bb6e3" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 33807 } }
I20260924 01:00:47.571365 26668 raft_consensus.cc:374] T f4a30dc8ea3c4a15967d6a3502bd1f7b P 07fb13b3dddd4fa38aa66a2fe9e190c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 01:00:47.571429 26668 raft_consensus.cc:729] T f4a30dc8ea3c4a15967d6a3502bd1f7b P 07fb13b3dddd4fa38aa66a2fe9e190c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07fb13b3dddd4fa38aa66a2fe9e190c1, State: Initialized, Role: FOLLOWER
I20260924 01:00:47.571931 26668 consensus_queue.cc:260] T f4a30dc8ea3c4a15967d6a3502bd1f7b P 07fb13b3dddd4fa38aa66a2fe9e190c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1dc17edd5a4bbc8496492c174fcb8e" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 39923 } } peers { permanent_uuid: "07fb13b3dddd4fa38aa66a2fe9e190c1" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35195 } } peers { permanent_uuid: "f142e4eefd1e40f49ecda7adf57bb6e3" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 33807 } }
I20260924 01:00:47.573535 26668 ts_tablet_manager.cc:1428] T f4a30dc8ea3c4a15967d6a3502bd1f7b P 07fb13b3dddd4fa38aa66a2fe9e190c1: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260924 01:00:47.596441 26670 raft_consensus.cc:482] T f4a30dc8ea3c4a15967d6a3502bd1f7b P f142e4eefd1e40f49ecda7adf57bb6e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 01:00:47.596704 26670 raft_consensus.cc:504] T f4a30dc8ea3c4a15967d6a3502bd1f7b P f142e4eefd1e40f49ecda7adf57bb6e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1dc17edd5a4bbc8496492c174fcb8e" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 39923 } } peers { permanent_uuid: "07fb13b3dddd4fa38aa66a2fe9e190c1" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35195 } } peers { permanent_uuid: "f142e4eefd1e40f49ecda7adf57bb6e3" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 33807 } }
I20260924 01:00:47.598407 26670 leader_election.cc:288] T f4a30dc8ea3c4a15967d6a3502bd1f7b P f142e4eefd1e40f49ecda7adf57bb6e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee1dc17edd5a4bbc8496492c174fcb8e (127.8.64.131:39923), 07fb13b3dddd4fa38aa66a2fe9e190c1 (127.8.64.129:35195)
I20260924 01:00:47.605218 26662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:47.599070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:44195 (local address 127.8.64.129:35195)
0924 01:00:47.599302 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:47.599309 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:47.599894 (+ 585us) server_negotiation.cc:407] Connection header received
0924 01:00:47.600084 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:47.600088 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:47.600160 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:47.600251 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 01:00:47.600863 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:47.601563 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:47.603551 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:47.604307 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:47.604503 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:47.604576 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:47.604838 (+ 262us) server_negotiation.cc:299] Negotiation successful
0924 01:00:47.604961 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260924 01:00:47.605273 26661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:47.598826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:33367 (local address 127.8.64.131:39923)
0924 01:00:47.599041 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:47.599047 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:47.599810 (+ 763us) server_negotiation.cc:407] Connection header received
0924 01:00:47.600020 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:47.600031 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:47.600092 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:47.600187 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 01:00:47.600798 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:47.601511 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:47.603572 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:47.604345 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:47.604565 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:47.604651 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:47.604902 (+ 251us) server_negotiation.cc:299] Negotiation successful
0924 01:00:47.605024 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260924 01:00:47.605516 26673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:47.598826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.131:39923 (local address 127.8.64.130:33367)
0924 01:00:47.599697 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:47.599723 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:47.599884 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:47.600231 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:47.600236 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:47.600266 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 01:00:47.600639 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:47.600648 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:47.601654 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:47.601662 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:47.603350 (+ 1688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:47.603362 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:47.604547 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:47.604555 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:47.604743 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:47.604765 (+ 22us) client_negotiation.cc:769] Sending connection context
0924 01:00:47.604873 (+ 108us) client_negotiation.cc:240] Negotiation successful
0924 01:00:47.605024 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":696,"mutex_wait_us":94,"thread_start_us":414,"threads_started":1}
I20260924 01:00:47.605693 26674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:47.598886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:35195 (local address 127.8.64.130:44195)
0924 01:00:47.599800 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:47.599814 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:47.599922 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:47.600276 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:47.600281 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:47.600303 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 01:00:47.600693 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:47.600701 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:47.601693 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:47.601699 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:47.603345 (+ 1646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:47.603357 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:47.604500 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:47.604506 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:47.604679 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:47.604702 (+ 23us) client_negotiation.cc:769] Sending connection context
0924 01:00:47.604810 (+ 108us) client_negotiation.cc:240] Negotiation successful
0924 01:00:47.604936 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":820,"mutex_wait_us":96,"thread_start_us":635,"threads_started":1}
I20260924 01:00:47.606698 26601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4a30dc8ea3c4a15967d6a3502bd1f7b" candidate_uuid: "f142e4eefd1e40f49ecda7adf57bb6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee1dc17edd5a4bbc8496492c174fcb8e" is_pre_election: true
I20260924 01:00:47.606732 26340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4a30dc8ea3c4a15967d6a3502bd1f7b" candidate_uuid: "f142e4eefd1e40f49ecda7adf57bb6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07fb13b3dddd4fa38aa66a2fe9e190c1" is_pre_election: true
I20260924 01:00:47.607244 26340 raft_consensus.cc:2454] T f4a30dc8ea3c4a15967d6a3502bd1f7b P 07fb13b3dddd4fa38aa66a2fe9e190c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f142e4eefd1e40f49ecda7adf57bb6e3 in term 0.
I20260924 01:00:47.607226 26601 raft_consensus.cc:2454] T f4a30dc8ea3c4a15967d6a3502bd1f7b P ee1dc17edd5a4bbc8496492c174fcb8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f142e4eefd1e40f49ecda7adf57bb6e3 in term 0.
I20260924 01:00:47.607914 26405 leader_election.cc:302] T f4a30dc8ea3c4a15967d6a3502bd1f7b P f142e4eefd1e40f49ecda7adf57bb6e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07fb13b3dddd4fa38aa66a2fe9e190c1, f142e4eefd1e40f49ecda7adf57bb6e3; no voters:
I20260924 01:00:47.608352 26670 raft_consensus.cc:2781] T f4a30dc8ea3c4a15967d6a3502bd1f7b P f142e4eefd1e40f49ecda7adf57bb6e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 01:00:47.608508 26670 raft_consensus.cc:482] T f4a30dc8ea3c4a15967d6a3502bd1f7b P f142e4eefd1e40f49ecda7adf57bb6e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 01:00:47.608613 26670 raft_consensus.cc:3037] T f4a30dc8ea3c4a15967d6a3502bd1f7b P f142e4eefd1e40f49ecda7adf57bb6e3 [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:00:47.611784 26670 raft_consensus.cc:504] T f4a30dc8ea3c4a15967d6a3502bd1f7b P f142e4eefd1e40f49ecda7adf57bb6e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1dc17edd5a4bbc8496492c174fcb8e" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 39923 } } peers { permanent_uuid: "07fb13b3dddd4fa38aa66a2fe9e190c1" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35195 } } peers { permanent_uuid: "f142e4eefd1e40f49ecda7adf57bb6e3" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 33807 } }
I20260924 01:00:47.612730 26670 leader_election.cc:288] T f4a30dc8ea3c4a15967d6a3502bd1f7b P f142e4eefd1e40f49ecda7adf57bb6e3 [CANDIDATE]: Term 1 election: Requested vote from peers ee1dc17edd5a4bbc8496492c174fcb8e (127.8.64.131:39923), 07fb13b3dddd4fa38aa66a2fe9e190c1 (127.8.64.129:35195)
I20260924 01:00:47.613287 26601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4a30dc8ea3c4a15967d6a3502bd1f7b" candidate_uuid: "f142e4eefd1e40f49ecda7adf57bb6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee1dc17edd5a4bbc8496492c174fcb8e"
I20260924 01:00:47.613438 26340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4a30dc8ea3c4a15967d6a3502bd1f7b" candidate_uuid: "f142e4eefd1e40f49ecda7adf57bb6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07fb13b3dddd4fa38aa66a2fe9e190c1"
I20260924 01:00:47.613570 26601 raft_consensus.cc:3037] T f4a30dc8ea3c4a15967d6a3502bd1f7b P ee1dc17edd5a4bbc8496492c174fcb8e [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:00:47.613711 26340 raft_consensus.cc:3037] T f4a30dc8ea3c4a15967d6a3502bd1f7b P 07fb13b3dddd4fa38aa66a2fe9e190c1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:00:47.616219 26340 raft_consensus.cc:2454] T f4a30dc8ea3c4a15967d6a3502bd1f7b P 07fb13b3dddd4fa38aa66a2fe9e190c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f142e4eefd1e40f49ecda7adf57bb6e3 in term 1.
I20260924 01:00:47.616264 26601 raft_consensus.cc:2454] T f4a30dc8ea3c4a15967d6a3502bd1f7b P ee1dc17edd5a4bbc8496492c174fcb8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f142e4eefd1e40f49ecda7adf57bb6e3 in term 1.
I20260924 01:00:47.616742 26404 leader_election.cc:302] T f4a30dc8ea3c4a15967d6a3502bd1f7b P f142e4eefd1e40f49ecda7adf57bb6e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ee1dc17edd5a4bbc8496492c174fcb8e, f142e4eefd1e40f49ecda7adf57bb6e3; no voters:
I20260924 01:00:47.617168 26670 raft_consensus.cc:2781] T f4a30dc8ea3c4a15967d6a3502bd1f7b P f142e4eefd1e40f49ecda7adf57bb6e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 01:00:47.617624 26670 raft_consensus.cc:686] T f4a30dc8ea3c4a15967d6a3502bd1f7b P f142e4eefd1e40f49ecda7adf57bb6e3 [term 1 LEADER]: Becoming Leader. State: Replica: f142e4eefd1e40f49ecda7adf57bb6e3, State: Running, Role: LEADER
I20260924 01:00:47.618116 26670 consensus_queue.cc:237] T f4a30dc8ea3c4a15967d6a3502bd1f7b P f142e4eefd1e40f49ecda7adf57bb6e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1dc17edd5a4bbc8496492c174fcb8e" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 39923 } } peers { permanent_uuid: "07fb13b3dddd4fa38aa66a2fe9e190c1" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35195 } } peers { permanent_uuid: "f142e4eefd1e40f49ecda7adf57bb6e3" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 33807 } }
I20260924 01:00:47.623024 26198 catalog_manager.cc:5495] T f4a30dc8ea3c4a15967d6a3502bd1f7b P f142e4eefd1e40f49ecda7adf57bb6e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to f142e4eefd1e40f49ecda7adf57bb6e3 (127.8.64.130). New cstate: current_term: 1 leader_uuid: "f142e4eefd1e40f49ecda7adf57bb6e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1dc17edd5a4bbc8496492c174fcb8e" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 39923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07fb13b3dddd4fa38aa66a2fe9e190c1" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f142e4eefd1e40f49ecda7adf57bb6e3" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 33807 } health_report { overall_health: HEALTHY } } }
I20260924 01:00:47.650822 26663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:47.645408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40462 (local address 127.8.64.130:33807)
0924 01:00:47.645623 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:47.645628 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:47.645646 (+ 18us) server_negotiation.cc:407] Connection header received
0924 01:00:47.645795 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:47.645799 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:47.645892 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:47.646018 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 01:00:47.646390 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:47.647097 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:47.648478 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:47.648973 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:47.649069 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:47.649522 (+ 453us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 01:00:47.650139 (+ 617us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 01:00:47.650248 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:47.650449 (+ 201us) server_negotiation.cc:299] Negotiation successful
0924 01:00:47.650512 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260924 01:00:47.659024 26340 raft_consensus.cc:1261] T f4a30dc8ea3c4a15967d6a3502bd1f7b P 07fb13b3dddd4fa38aa66a2fe9e190c1 [term 1 FOLLOWER]: Refusing update from remote peer f142e4eefd1e40f49ecda7adf57bb6e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 01:00:47.659032 26601 raft_consensus.cc:1261] T f4a30dc8ea3c4a15967d6a3502bd1f7b P ee1dc17edd5a4bbc8496492c174fcb8e [term 1 FOLLOWER]: Refusing update from remote peer f142e4eefd1e40f49ecda7adf57bb6e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 01:00:47.660009 26670 consensus_queue.cc:1035] T f4a30dc8ea3c4a15967d6a3502bd1f7b P f142e4eefd1e40f49ecda7adf57bb6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee1dc17edd5a4bbc8496492c174fcb8e" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 39923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 01:00:47.660342 26675 consensus_queue.cc:1035] T f4a30dc8ea3c4a15967d6a3502bd1f7b P f142e4eefd1e40f49ecda7adf57bb6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07fb13b3dddd4fa38aa66a2fe9e190c1" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 35195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 01:00:47.670065 26678 mvcc.cc:205] Tried to move back new op lower bound from 7332706908801601536 to 7332706908647907328. Current Snapshot: MvccSnapshot[applied={T|T < 7332706908801601536}]
I20260924 01:00:47.670387 26679 mvcc.cc:205] Tried to move back new op lower bound from 7332706908801601536 to 7332706908647907328. Current Snapshot: MvccSnapshot[applied={T|T < 7332706908801601536}]
I20260924 01:00:47.686900 26198 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46972:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 01:00:47.687198 26198 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46972:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 01:00:47.690817 26198 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0e04792fb57c409cb3ffc914ede22e92: Sending DeleteTablet for 3 replicas of tablet f4a30dc8ea3c4a15967d6a3502bd1f7b
I20260924 01:00:47.692037 26320 tablet_service.cc:1514] Processing DeleteTablet for tablet f4a30dc8ea3c4a15967d6a3502bd1f7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 01:00:47 UTC) from {username='slave'} at 127.0.0.1:48642
I20260924 01:00:47.692072 26581 tablet_service.cc:1514] Processing DeleteTablet for tablet f4a30dc8ea3c4a15967d6a3502bd1f7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 01:00:47 UTC) from {username='slave'} at 127.0.0.1:42360
I20260924 01:00:47.692323 26451 tablet_service.cc:1514] Processing DeleteTablet for tablet f4a30dc8ea3c4a15967d6a3502bd1f7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 01:00:47 UTC) from {username='slave'} at 127.0.0.1:40452
I20260924 01:00:47.699580 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 26258
I20260924 01:00:47.702788 26689 tablet_replica.cc:330] T f4a30dc8ea3c4a15967d6a3502bd1f7b P 07fb13b3dddd4fa38aa66a2fe9e190c1: stopping tablet replica
I20260924 01:00:47.703274 26689 raft_consensus.cc:2229] T f4a30dc8ea3c4a15967d6a3502bd1f7b P 07fb13b3dddd4fa38aa66a2fe9e190c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 01:00:47.703814 26689 raft_consensus.cc:2258] T f4a30dc8ea3c4a15967d6a3502bd1f7b P 07fb13b3dddd4fa38aa66a2fe9e190c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 01:00:47.708001 26689 ts_tablet_manager.cc:1905] T f4a30dc8ea3c4a15967d6a3502bd1f7b P 07fb13b3dddd4fa38aa66a2fe9e190c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 01:00:47.713429 26688 tablet_replica.cc:330] T f4a30dc8ea3c4a15967d6a3502bd1f7b P ee1dc17edd5a4bbc8496492c174fcb8e: stopping tablet replica
I20260924 01:00:47.714021 26688 raft_consensus.cc:2229] T f4a30dc8ea3c4a15967d6a3502bd1f7b P ee1dc17edd5a4bbc8496492c174fcb8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 01:00:47.714581 26688 raft_consensus.cc:2258] T f4a30dc8ea3c4a15967d6a3502bd1f7b P ee1dc17edd5a4bbc8496492c174fcb8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 01:00:47.718533 26690 tablet_replica.cc:330] T f4a30dc8ea3c4a15967d6a3502bd1f7b P f142e4eefd1e40f49ecda7adf57bb6e3: stopping tablet replica
I20260924 01:00:47.718930 26690 raft_consensus.cc:2229] T f4a30dc8ea3c4a15967d6a3502bd1f7b P f142e4eefd1e40f49ecda7adf57bb6e3 [term 1 LEADER]: Raft consensus shutting down.
I20260924 01:00:47.719521 26690 raft_consensus.cc:2258] T f4a30dc8ea3c4a15967d6a3502bd1f7b P f142e4eefd1e40f49ecda7adf57bb6e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 01:00:47.719997 26689 ts_tablet_manager.cc:1918] T f4a30dc8ea3c4a15967d6a3502bd1f7b P 07fb13b3dddd4fa38aa66a2fe9e190c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 01:00:47.720185 26689 log.cc:1196] T f4a30dc8ea3c4a15967d6a3502bd1f7b P 07fb13b3dddd4fa38aa66a2fe9e190c1: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-0/wal/wals/f4a30dc8ea3c4a15967d6a3502bd1f7b
I20260924 01:00:47.720800 26689 ts_tablet_manager.cc:1939] T f4a30dc8ea3c4a15967d6a3502bd1f7b P 07fb13b3dddd4fa38aa66a2fe9e190c1: Deleting consensus metadata
I20260924 01:00:47.722782 26184 catalog_manager.cc:4841] TS 07fb13b3dddd4fa38aa66a2fe9e190c1 (127.8.64.129:35195): tablet f4a30dc8ea3c4a15967d6a3502bd1f7b (table test-table [id=ff1c8053f1de44ebb8e35e1f686082b2]) successfully deleted
I20260924 01:00:47.723321 26690 ts_tablet_manager.cc:1905] T f4a30dc8ea3c4a15967d6a3502bd1f7b P f142e4eefd1e40f49ecda7adf57bb6e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 01:00:47.724571 26688 ts_tablet_manager.cc:1905] T f4a30dc8ea3c4a15967d6a3502bd1f7b P ee1dc17edd5a4bbc8496492c174fcb8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 01:00:47.729092 26662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:47.702124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48654 (local address 127.8.64.129:35195)
0924 01:00:47.702396 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:47.702409 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:47.712087 (+ 9678us) server_negotiation.cc:407] Connection header received
0924 01:00:47.712637 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:47.712646 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:47.712774 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:47.712866 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:47.713755 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:47.715231 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:47.719927 (+ 4696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:47.720583 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:47.720676 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:47.727336 (+ 6660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:47.727348 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:47.727365 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:00:47.728119 (+ 754us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:47.728185 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:00:47.728215 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:47.728313 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:47.728577 (+ 264us) server_negotiation.cc:299] Negotiation successful
0924 01:00:47.728701 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260924 01:00:47.729410 26690 ts_tablet_manager.cc:1918] T f4a30dc8ea3c4a15967d6a3502bd1f7b P f142e4eefd1e40f49ecda7adf57bb6e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 01:00:47.729590 26690 log.cc:1196] T f4a30dc8ea3c4a15967d6a3502bd1f7b P f142e4eefd1e40f49ecda7adf57bb6e3: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-1/wal/wals/f4a30dc8ea3c4a15967d6a3502bd1f7b
I20260924 01:00:47.730374 26690 ts_tablet_manager.cc:1939] T f4a30dc8ea3c4a15967d6a3502bd1f7b P f142e4eefd1e40f49ecda7adf57bb6e3: Deleting consensus metadata
I20260924 01:00:47.730744 26380 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 01:00:47.731808 26688 ts_tablet_manager.cc:1918] T f4a30dc8ea3c4a15967d6a3502bd1f7b P ee1dc17edd5a4bbc8496492c174fcb8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 01:00:47.732103 26688 log.cc:1196] T f4a30dc8ea3c4a15967d6a3502bd1f7b P ee1dc17edd5a4bbc8496492c174fcb8e: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790211343099518-8450-0/minicluster-data/ts-2/wal/wals/f4a30dc8ea3c4a15967d6a3502bd1f7b
I20260924 01:00:47.732201 26186 catalog_manager.cc:4841] TS f142e4eefd1e40f49ecda7adf57bb6e3 (127.8.64.130:33807): tablet f4a30dc8ea3c4a15967d6a3502bd1f7b (table test-table [id=ff1c8053f1de44ebb8e35e1f686082b2]) successfully deleted
I20260924 01:00:47.732919 26688 ts_tablet_manager.cc:1939] T f4a30dc8ea3c4a15967d6a3502bd1f7b P ee1dc17edd5a4bbc8496492c174fcb8e: Deleting consensus metadata
I20260924 01:00:47.735070 26183 catalog_manager.cc:4841] TS ee1dc17edd5a4bbc8496492c174fcb8e (127.8.64.131:39923): tablet f4a30dc8ea3c4a15967d6a3502bd1f7b (table test-table [id=ff1c8053f1de44ebb8e35e1f686082b2]) successfully deleted
I20260924 01:00:47.825104 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 26258
I20260924 01:00:47.850080 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 26389
I20260924 01:00:47.858336 26663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:47.850970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40476 (local address 127.8.64.130:33807)
0924 01:00:47.851191 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:47.851198 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:47.851220 (+ 22us) server_negotiation.cc:407] Connection header received
0924 01:00:47.851292 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:47.851297 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:47.851397 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:47.851518 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:47.851954 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:47.853177 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:47.854600 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:47.855276 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:47.855414 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:47.856610 (+ 1196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:47.856622 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:47.856641 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:00:47.857476 (+ 835us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:47.857535 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:00:47.857561 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:47.857674 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:47.857851 (+ 177us) server_negotiation.cc:299] Negotiation successful
0924 01:00:47.857947 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260924 01:00:47.859058 26511 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 01:00:47.973060 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 26389
I20260924 01:00:47.992427 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 26519
I20260924 01:00:47.999171 26661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:47.993206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42368 (local address 127.8.64.131:39923)
0924 01:00:47.993371 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:47.993375 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:47.993436 (+ 61us) server_negotiation.cc:407] Connection header received
0924 01:00:47.993608 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:47.993612 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:47.993692 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:47.993782 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:47.994155 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:47.994946 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:47.996289 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:47.996684 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:47.996779 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:47.997781 (+ 1002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:47.997789 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:47.997803 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:00:47.998528 (+ 725us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:47.998578 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:00:47.998604 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:47.998692 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:47.998840 (+ 148us) server_negotiation.cc:299] Negotiation successful
0924 01:00:47.998929 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260924 01:00:47.999799 26641 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 01:00:48.087899 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 26519
I20260924 01:00:48.104797 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 26168
I20260924 01:00:48.106016 26229 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 01:00:48.217141 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 26168
2026-09-24T01:00:48Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2790 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 01:00:48 dist-test-slave-mmhs krb5kdc[26698](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 01:00:48 dist-test-slave-mmhs krb5kdc[26698](info): set up 2 sockets
Sep 24 01:00:48 dist-test-slave-mmhs krb5kdc[26698](info): commencing operation
krb5kdc: starting...
W20260924 01:00:49.324035 8450 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.003s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Sep 24 01:00:49 dist-test-slave-mmhs krb5kdc[26698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T01:00:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T01:00:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.64.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
I20260924 01:00:49.585115 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:40783
--webserver_interface=127.8.64.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:40271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:40783
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 01:00:49.907956 26711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:00:49.908245 26711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:00:49.908334 26711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:00:49.915473 26711 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 01:00:49.915593 26711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:00:49.915640 26711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:00:49.915699 26711 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 01:00:49.915742 26711 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 01:00:49.925495 26711 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:40271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:40783
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:40783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:00:49.926813 26711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:00:49.928093 26711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:00:49.940088 26716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:49.940093 26719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:49.940081 26717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:00:49.941515 26711 server_base.cc:1034] running on GCE node
I20260924 01:00:49.942287 26711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:00:49.943804 26711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:00:49.945015 26711 hybrid_clock.cc:648] HybridClock initialized: now 1790211649944974 us; error 29 us; skew 500 ppm
I20260924 01:00:49.945329 26711 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 01:00:49 dist-test-slave-mmhs krb5kdc[26698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211649, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 01:00:49.948379 26711 init.cc:377] Logged in from keytab as kudu/127.8.64.190@KRBTEST.COM (short username kudu)
I20260924 01:00:49.949788 26711 webserver.cc:460] Webserver started at http://127.8.64.190:38107/ using document root <none> and password file <none>
I20260924 01:00:49.950484 26711 fs_manager.cc:359] Metadata directory not provided
I20260924 01:00:49.950600 26711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:00:49.950845 26711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:00:49.953718 26711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "174a27e379ae451684f8f51ec3d86778"
format_stamp: "Formatted at 2026-09-24 01:00:49 on dist-test-slave-mmhs"
server_key: "dd4ed6b1c89f048a3b32e7a5852da6d3"
server_key_iv: "f97049888ac1032565451f5beccac003"
server_key_version: "encryptionkey@0"
I20260924 01:00:49.954492 26711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "174a27e379ae451684f8f51ec3d86778"
format_stamp: "Formatted at 2026-09-24 01:00:49 on dist-test-slave-mmhs"
server_key: "dd4ed6b1c89f048a3b32e7a5852da6d3"
server_key_iv: "f97049888ac1032565451f5beccac003"
server_key_version: "encryptionkey@0"
I20260924 01:00:49.959760 26711 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260924 01:00:49.963295 26726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:49.964601 26711 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260924 01:00:49.964744 26711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "174a27e379ae451684f8f51ec3d86778"
format_stamp: "Formatted at 2026-09-24 01:00:49 on dist-test-slave-mmhs"
server_key: "dd4ed6b1c89f048a3b32e7a5852da6d3"
server_key_iv: "f97049888ac1032565451f5beccac003"
server_key_version: "encryptionkey@0"
I20260924 01:00:49.964905 26711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:00:49.985090 26711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:00:49.989387 26711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:00:49.989621 26711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:00:50.007920 26711 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.190:40783
I20260924 01:00:50.007958 26777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.190:40783 every 8 connection(s)
I20260924 01:00:50.009289 26711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 01:00:50.013016 26778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:00:50.019323 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 26711
I20260924 01:00:50.019511 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 01:00:50.019989 8450 external_mini_cluster.cc:1428] Setting key f764fc9be2b52ea01118cd8faf078cf9
I20260924 01:00:50.024180 26778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 174a27e379ae451684f8f51ec3d86778: Bootstrap starting.
I20260924 01:00:50.027930 26778 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 174a27e379ae451684f8f51ec3d86778: Neither blocks nor log segments found. Creating new log.
I20260924 01:00:50.029156 26778 log.cc:824] T 00000000000000000000000000000000 P 174a27e379ae451684f8f51ec3d86778: Log is configured to *not* fsync() on all Append() calls
I20260924 01:00:50.032262 26778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 174a27e379ae451684f8f51ec3d86778: No bootstrap required, opened a new log
Sep 24 01:00:50 dist-test-slave-mmhs krb5kdc[26698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 01:00:50.041555 26778 raft_consensus.cc:348] T 00000000000000000000000000000000 P 174a27e379ae451684f8f51ec3d86778 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "174a27e379ae451684f8f51ec3d86778" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 40783 } }
I20260924 01:00:50.041990 26778 raft_consensus.cc:374] T 00000000000000000000000000000000 P 174a27e379ae451684f8f51ec3d86778 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 01:00:50.042112 26778 raft_consensus.cc:729] T 00000000000000000000000000000000 P 174a27e379ae451684f8f51ec3d86778 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 174a27e379ae451684f8f51ec3d86778, State: Initialized, Role: FOLLOWER
I20260924 01:00:50.042870 26778 consensus_queue.cc:260] T 00000000000000000000000000000000 P 174a27e379ae451684f8f51ec3d86778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "174a27e379ae451684f8f51ec3d86778" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 40783 } }
I20260924 01:00:50.043170 26778 raft_consensus.cc:388] T 00000000000000000000000000000000 P 174a27e379ae451684f8f51ec3d86778 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 01:00:50.043319 26778 raft_consensus.cc:482] T 00000000000000000000000000000000 P 174a27e379ae451684f8f51ec3d86778 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 01:00:50.043520 26778 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 174a27e379ae451684f8f51ec3d86778 [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:00:50.045038 26781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:50.021917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33848 (local address 127.8.64.190:40783)
0924 01:00:50.022662 (+ 745us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:50.022677 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:50.022726 (+ 49us) server_negotiation.cc:407] Connection header received
0924 01:00:50.025071 (+ 2345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:50.025098 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:50.025561 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:50.026028 (+ 467us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:50.027413 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:50.029411 (+ 1998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:50.031100 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:50.032107 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:50.032291 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:50.036498 (+ 4207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:50.036510 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:50.036533 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:00:50.036588 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:50.040717 (+ 4129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:50.041422 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:50.041430 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:50.041438 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:50.041657 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:50.042054 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:50.042060 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:50.042065 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:50.042776 (+ 711us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:50.043116 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:50.043563 (+ 447us) server_negotiation.cc:299] Negotiation successful
0924 01:00:50.043989 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":331,"threads_started":1}
I20260924 01:00:50.046038 26778 raft_consensus.cc:504] T 00000000000000000000000000000000 P 174a27e379ae451684f8f51ec3d86778 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "174a27e379ae451684f8f51ec3d86778" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 40783 } }
I20260924 01:00:50.046638 26778 leader_election.cc:302] T 00000000000000000000000000000000 P 174a27e379ae451684f8f51ec3d86778 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 174a27e379ae451684f8f51ec3d86778; no voters:
I20260924 01:00:50.047284 26778 leader_election.cc:288] T 00000000000000000000000000000000 P 174a27e379ae451684f8f51ec3d86778 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 01:00:50.047397 26783 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 174a27e379ae451684f8f51ec3d86778 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 01:00:50.048686 26783 raft_consensus.cc:686] T 00000000000000000000000000000000 P 174a27e379ae451684f8f51ec3d86778 [term 1 LEADER]: Becoming Leader. State: Replica: 174a27e379ae451684f8f51ec3d86778, State: Running, Role: LEADER
I20260924 01:00:50.049263 26778 sys_catalog.cc:564] T 00000000000000000000000000000000 P 174a27e379ae451684f8f51ec3d86778 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 01:00:50.049232 26783 consensus_queue.cc:237] T 00000000000000000000000000000000 P 174a27e379ae451684f8f51ec3d86778 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "174a27e379ae451684f8f51ec3d86778" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 40783 } }
I20260924 01:00:50.052322 26785 sys_catalog.cc:455] T 00000000000000000000000000000000 P 174a27e379ae451684f8f51ec3d86778 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 174a27e379ae451684f8f51ec3d86778. Latest consensus state: current_term: 1 leader_uuid: "174a27e379ae451684f8f51ec3d86778" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "174a27e379ae451684f8f51ec3d86778" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 40783 } } }
I20260924 01:00:50.052762 26785 sys_catalog.cc:458] T 00000000000000000000000000000000 P 174a27e379ae451684f8f51ec3d86778 [sys.catalog]: This master's current role is: LEADER
I20260924 01:00:50.053246 26784 sys_catalog.cc:455] T 00000000000000000000000000000000 P 174a27e379ae451684f8f51ec3d86778 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "174a27e379ae451684f8f51ec3d86778" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "174a27e379ae451684f8f51ec3d86778" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 40783 } } }
I20260924 01:00:50.053565 26784 sys_catalog.cc:458] T 00000000000000000000000000000000 P 174a27e379ae451684f8f51ec3d86778 [sys.catalog]: This master's current role is: LEADER
I20260924 01:00:50.055308 26792 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 01:00:50.059446 26792 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 01:00:50.067692 26792 catalog_manager.cc:1347] Generated new cluster ID: 71dbe0f1f063433fa09ccd93c51870c8
I20260924 01:00:50.067801 26792 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 01:00:50.081036 26792 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 01:00:50.082381 26792 catalog_manager.cc:1504] Loading token signing keys...
I20260924 01:00:50.089031 26792 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 174a27e379ae451684f8f51ec3d86778: Generated new TSK 0
I20260924 01:00:50.090032 26792 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.64.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab.
Entry for principal kudu/127.8.64.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab.
I20260924 01:00:50.153272 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.129:0
--local_ip_for_outbound_sockets=127.8.64.129
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:40783
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab
--principal=kudu/127.8.64.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:40271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 01:00:50.485329 26804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:00:50.485589 26804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:00:50.485770 26804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:00:50.493070 26804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:00:50.493173 26804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:00:50.493342 26804 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.129
I20260924 01:00:50.502849 26804 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:40271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab
--principal=kudu/127.8.64.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:40783
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.129
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:00:50.504058 26804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:00:50.505363 26804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:00:50.518431 26809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:50.518508 26812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:50.518471 26810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:00:50.519543 26804 server_base.cc:1034] running on GCE node
I20260924 01:00:50.520151 26804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:00:50.521070 26804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:00:50.522339 26804 hybrid_clock.cc:648] HybridClock initialized: now 1790211650522256 us; error 59 us; skew 500 ppm
I20260924 01:00:50.522665 26804 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 01:00:50 dist-test-slave-mmhs krb5kdc[26698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211650, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 01:00:50.525714 26804 init.cc:377] Logged in from keytab as kudu/127.8.64.129@KRBTEST.COM (short username kudu)
I20260924 01:00:50.527158 26804 webserver.cc:460] Webserver started at http://127.8.64.129:42577/ using document root <none> and password file <none>
I20260924 01:00:50.527757 26804 fs_manager.cc:359] Metadata directory not provided
I20260924 01:00:50.527853 26804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:00:50.528096 26804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:00:50.530951 26804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-0/data/instance:
uuid: "44bffb5db78d4dc79d408b49a9a22c07"
format_stamp: "Formatted at 2026-09-24 01:00:50 on dist-test-slave-mmhs"
server_key: "5d2cb7327396f283f193b25336bf3643"
server_key_iv: "34e2b1dd0ebf67a63dadf5ef8d1954d3"
server_key_version: "encryptionkey@0"
I20260924 01:00:50.531662 26804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance:
uuid: "44bffb5db78d4dc79d408b49a9a22c07"
format_stamp: "Formatted at 2026-09-24 01:00:50 on dist-test-slave-mmhs"
server_key: "5d2cb7327396f283f193b25336bf3643"
server_key_iv: "34e2b1dd0ebf67a63dadf5ef8d1954d3"
server_key_version: "encryptionkey@0"
I20260924 01:00:50.536583 26804 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260924 01:00:50.539809 26819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:50.541047 26804 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260924 01:00:50.541173 26804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-0/wal
uuid: "44bffb5db78d4dc79d408b49a9a22c07"
format_stamp: "Formatted at 2026-09-24 01:00:50 on dist-test-slave-mmhs"
server_key: "5d2cb7327396f283f193b25336bf3643"
server_key_iv: "34e2b1dd0ebf67a63dadf5ef8d1954d3"
server_key_version: "encryptionkey@0"
I20260924 01:00:50.541324 26804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:00:50.562229 26804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:00:50.566602 26804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:00:50.566859 26804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:00:50.567767 26804 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 01:00:50.569248 26804 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 01:00:50.569322 26804 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:50.569478 26804 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 01:00:50.569537 26804 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:50.602913 26804 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.129:34229
I20260924 01:00:50.602993 26931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.129:34229 every 8 connection(s)
I20260924 01:00:50.604177 26804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
I20260924 01:00:50.609159 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 26804
I20260924 01:00:50.609333 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance
I20260924 01:00:50.609874 8450 external_mini_cluster.cc:1428] Setting key 77069d1859bcd8a9dbb998791c951c69
Sep 24 01:00:50 dist-test-slave-mmhs krb5kdc[26698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211650, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 01:00:50.626156 26934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:50.606726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:50861 (local address 127.8.64.190:40783)
0924 01:00:50.607153 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:50.607157 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:50.608092 (+ 935us) server_negotiation.cc:407] Connection header received
0924 01:00:50.610507 (+ 2415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:50.610514 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:50.610611 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:50.610733 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:50.612540 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:50.613244 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:50.615759 (+ 2515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:50.616157 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:50.616251 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:50.620292 (+ 4041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:50.620301 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:50.620309 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:00:50.620354 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:50.623266 (+ 2912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:50.623935 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:50.623940 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:50.623951 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:50.624069 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:50.624491 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:50.624496 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:50.624500 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:50.624767 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:50.624922 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:50.625586 (+ 664us) server_negotiation.cc:299] Negotiation successful
0924 01:00:50.625809 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":210,"threads_started":1}
I20260924 01:00:50.627194 26935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:50.607256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:40783 (local address 127.8.64.129:50861)
0924 01:00:50.607920 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:50.607967 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:50.610167 (+ 2200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:50.610984 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:50.610998 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:50.611593 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0924 01:00:50.612351 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:50.612381 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:50.613400 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:50.613409 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:50.615608 (+ 2199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:50.615625 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:50.616380 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:50.616389 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:50.616711 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:50.617418 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 01:00:50.617458 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0924 01:00:50.620071 (+ 2613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 01:00:50.623387 (+ 3316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:00:50.623393 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:00:50.623406 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:00:50.623803 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:00:50.624188 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:00:50.624194 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:00:50.624197 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:00:50.624371 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:00:50.624891 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 01:00:50.624900 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 01:00:50.625217 (+ 317us) client_negotiation.cc:769] Sending connection context
0924 01:00:50.625469 (+ 252us) client_negotiation.cc:240] Negotiation successful
0924 01:00:50.625965 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":268,"threads_started":1}
I20260924 01:00:50.628742 26932 heartbeater.cc:347] Connected to a master server at 127.8.64.190:40783
I20260924 01:00:50.629034 26932 heartbeater.cc:464] Registering TS with master...
I20260924 01:00:50.629716 26932 heartbeater.cc:511] Master 127.8.64.190:40783 requested a full tablet report, sending...
I20260924 01:00:50.632035 26743 ts_manager.cc:194] Registered new tserver with Master: 44bffb5db78d4dc79d408b49a9a22c07 (127.8.64.129:34229)
I20260924 01:00:50.634233 26743 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.129@KRBTEST.COM'} at 127.8.64.129:50861
WARNING: no policy specified for kudu/127.8.64.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab.
Entry for principal kudu/127.8.64.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab.
I20260924 01:00:50.669044 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.130:0
--local_ip_for_outbound_sockets=127.8.64.130
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:40783
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab
--principal=kudu/127.8.64.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:40271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 01:00:50.992745 26938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:00:50.993031 26938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:00:50.993125 26938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:00:51.000360 26938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:00:51.000475 26938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:00:51.000589 26938 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.130
I20260924 01:00:51.010334 26938 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:40271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab
--principal=kudu/127.8.64.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:40783
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.130
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:00:51.011646 26938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:00:51.012970 26938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:00:51.027182 26944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:51.027277 26943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:51.027246 26946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:00:51.027872 26938 server_base.cc:1034] running on GCE node
I20260924 01:00:51.028527 26938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:00:51.029420 26938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:00:51.030637 26938 hybrid_clock.cc:648] HybridClock initialized: now 1790211651030588 us; error 44 us; skew 500 ppm
I20260924 01:00:51.030941 26938 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 01:00:51 dist-test-slave-mmhs krb5kdc[26698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211651, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 01:00:51.034159 26938 init.cc:377] Logged in from keytab as kudu/127.8.64.130@KRBTEST.COM (short username kudu)
I20260924 01:00:51.035625 26938 webserver.cc:460] Webserver started at http://127.8.64.130:42225/ using document root <none> and password file <none>
I20260924 01:00:51.036265 26938 fs_manager.cc:359] Metadata directory not provided
I20260924 01:00:51.036363 26938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:00:51.036607 26938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:00:51.039501 26938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-1/data/instance:
uuid: "cd884c3bb146494594cbb14d53cf5a73"
format_stamp: "Formatted at 2026-09-24 01:00:51 on dist-test-slave-mmhs"
server_key: "dbe6a88fe6dbfd23ee7fb159f5a4a83d"
server_key_iv: "4740c324b2cd8ae03330cb7ec3afe89a"
server_key_version: "encryptionkey@0"
I20260924 01:00:51.040201 26938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance:
uuid: "cd884c3bb146494594cbb14d53cf5a73"
format_stamp: "Formatted at 2026-09-24 01:00:51 on dist-test-slave-mmhs"
server_key: "dbe6a88fe6dbfd23ee7fb159f5a4a83d"
server_key_iv: "4740c324b2cd8ae03330cb7ec3afe89a"
server_key_version: "encryptionkey@0"
I20260924 01:00:51.045323 26938 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260924 01:00:51.048936 26953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:51.050417 26938 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260924 01:00:51.050601 26938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-1/wal
uuid: "cd884c3bb146494594cbb14d53cf5a73"
format_stamp: "Formatted at 2026-09-24 01:00:51 on dist-test-slave-mmhs"
server_key: "dbe6a88fe6dbfd23ee7fb159f5a4a83d"
server_key_iv: "4740c324b2cd8ae03330cb7ec3afe89a"
server_key_version: "encryptionkey@0"
I20260924 01:00:51.050774 26938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:00:51.072448 26938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:00:51.076784 26938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:00:51.077083 26938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:00:51.078042 26938 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 01:00:51.079345 26938 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 01:00:51.079420 26938 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:51.079593 26938 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 01:00:51.079643 26938 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:51.112623 26938 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.130:35025
I20260924 01:00:51.112720 27065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.130:35025 every 8 connection(s)
I20260924 01:00:51.113909 26938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
I20260924 01:00:51.123150 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 26938
I20260924 01:00:51.123338 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance
I20260924 01:00:51.123919 8450 external_mini_cluster.cc:1428] Setting key f1cc82a5ccf1d709c4559b73df8e8217
Sep 24 01:00:51 dist-test-slave-mmhs krb5kdc[26698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211651, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.130@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 01:00:51.134238 26934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:51.116482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:55167 (local address 127.8.64.190:40783)
0924 01:00:51.116674 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:51.116678 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:51.117787 (+ 1109us) server_negotiation.cc:407] Connection header received
0924 01:00:51.119655 (+ 1868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:51.119659 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:51.119731 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:51.119818 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:51.121270 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:51.121981 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:51.124028 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:51.124543 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:51.124702 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:51.128675 (+ 3973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:51.128680 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:51.128683 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:00:51.128715 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:51.131333 (+ 2618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:51.132039 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:51.132051 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:51.132054 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:51.132174 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:51.132594 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:51.132599 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:51.132602 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:51.132858 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:51.133020 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:51.133679 (+ 659us) server_negotiation.cc:299] Negotiation successful
0924 01:00:51.133900 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260924 01:00:51.135218 27068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:51.116924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:40783 (local address 127.8.64.130:55167)
0924 01:00:51.117627 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:51.117666 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:51.119413 (+ 1747us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:51.120038 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:51.120045 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:51.120562 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0924 01:00:51.121119 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:51.121140 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:51.122127 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:51.122131 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:51.123824 (+ 1693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:51.123838 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:51.124765 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:51.124770 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:51.124993 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:51.125751 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 01:00:51.125797 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0924 01:00:51.128451 (+ 2654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 01:00:51.131474 (+ 3023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:00:51.131483 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:00:51.131501 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:00:51.131909 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:00:51.132285 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:00:51.132291 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:00:51.132297 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:00:51.132478 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:00:51.132983 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 01:00:51.133009 (+ 26us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 01:00:51.133324 (+ 315us) client_negotiation.cc:769] Sending connection context
0924 01:00:51.133580 (+ 256us) client_negotiation.cc:240] Negotiation successful
0924 01:00:51.134109 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":264,"threads_started":1}
I20260924 01:00:51.136714 27066 heartbeater.cc:347] Connected to a master server at 127.8.64.190:40783
I20260924 01:00:51.137035 27066 heartbeater.cc:464] Registering TS with master...
I20260924 01:00:51.137851 27066 heartbeater.cc:511] Master 127.8.64.190:40783 requested a full tablet report, sending...
I20260924 01:00:51.139511 26743 ts_manager.cc:194] Registered new tserver with Master: cd884c3bb146494594cbb14d53cf5a73 (127.8.64.130:35025)
I20260924 01:00:51.140336 26743 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.130@KRBTEST.COM'} at 127.8.64.130:55167
WARNING: no policy specified for kudu/127.8.64.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab.
Entry for principal kudu/127.8.64.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab.
I20260924 01:00:51.185322 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.131:0
--local_ip_for_outbound_sockets=127.8.64.131
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:40783
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab
--principal=kudu/127.8.64.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:40271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 01:00:51.509079 27071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:00:51.509341 27071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:00:51.509507 27071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:00:51.516712 27071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:00:51.516814 27071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:00:51.517023 27071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.131
I20260924 01:00:51.526523 27071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:40271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab
--principal=kudu/127.8.64.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:40783
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.131
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:00:51.527724 27071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:00:51.529019 27071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:00:51.542078 27077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:51.542038 27079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:51.542122 27076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:00:51.547887 27071 server_base.cc:1034] running on GCE node
I20260924 01:00:51.548584 27071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:00:51.549584 27071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:00:51.550791 27071 hybrid_clock.cc:648] HybridClock initialized: now 1790211651550713 us; error 67 us; skew 500 ppm
I20260924 01:00:51.551156 27071 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 01:00:51 dist-test-slave-mmhs krb5kdc[26698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211651, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 01:00:51.554806 27071 init.cc:377] Logged in from keytab as kudu/127.8.64.131@KRBTEST.COM (short username kudu)
I20260924 01:00:51.556491 27071 webserver.cc:460] Webserver started at http://127.8.64.131:35749/ using document root <none> and password file <none>
I20260924 01:00:51.557199 27071 fs_manager.cc:359] Metadata directory not provided
I20260924 01:00:51.557305 27071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:00:51.557579 27071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:00:51.560784 27071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-2/data/instance:
uuid: "47cdf4cc41e74168be0c2f281868d448"
format_stamp: "Formatted at 2026-09-24 01:00:51 on dist-test-slave-mmhs"
server_key: "30514ec202304211935dc49a30fce26f"
server_key_iv: "f0c63fd8073bbc247795aba685a50b38"
server_key_version: "encryptionkey@0"
I20260924 01:00:51.561596 27071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance:
uuid: "47cdf4cc41e74168be0c2f281868d448"
format_stamp: "Formatted at 2026-09-24 01:00:51 on dist-test-slave-mmhs"
server_key: "30514ec202304211935dc49a30fce26f"
server_key_iv: "f0c63fd8073bbc247795aba685a50b38"
server_key_version: "encryptionkey@0"
I20260924 01:00:51.567134 27071 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260924 01:00:51.570684 27086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:51.572003 27071 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260924 01:00:51.572173 27071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-2/wal
uuid: "47cdf4cc41e74168be0c2f281868d448"
format_stamp: "Formatted at 2026-09-24 01:00:51 on dist-test-slave-mmhs"
server_key: "30514ec202304211935dc49a30fce26f"
server_key_iv: "f0c63fd8073bbc247795aba685a50b38"
server_key_version: "encryptionkey@0"
I20260924 01:00:51.572378 27071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:00:51.589956 27071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:00:51.594208 27071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:00:51.594511 27071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:00:51.595460 27071 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 01:00:51.596786 27071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 01:00:51.596858 27071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:51.597030 27071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 01:00:51.597082 27071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:51.630282 27071 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.131:41031
I20260924 01:00:51.630430 27198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.131:41031 every 8 connection(s)
I20260924 01:00:51.631650 27071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
I20260924 01:00:51.632939 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 27071
I20260924 01:00:51.633141 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance
I20260924 01:00:51.633772 8450 external_mini_cluster.cc:1428] Setting key 1a7b64e8281a683bb977eeb01ad6c845
I20260924 01:00:51.637220 26932 heartbeater.cc:503] Master 127.8.64.190:40783 was elected leader, sending a full tablet report...
Sep 24 01:00:51 dist-test-slave-mmhs krb5kdc[26698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211651, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.131@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 01:00:51.654186 27201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:51.634663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:54403 (local address 127.8.64.190:40783)
0924 01:00:51.635131 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:51.635136 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:51.636511 (+ 1375us) server_negotiation.cc:407] Connection header received
0924 01:00:51.638947 (+ 2436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:51.638953 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:51.639049 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:51.639185 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:51.640762 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:51.641607 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:51.643353 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:51.643806 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:51.643960 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:51.647740 (+ 3780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:51.647746 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:51.647750 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:00:51.647785 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:51.650967 (+ 3182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:51.651743 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:51.651754 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:51.651756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:51.651889 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:51.652346 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:51.652357 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:51.652360 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:51.652630 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:51.652816 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:51.653541 (+ 725us) server_negotiation.cc:299] Negotiation successful
0924 01:00:51.653771 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":239,"threads_started":1}
I20260924 01:00:51.655267 27202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:51.635312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:40783 (local address 127.8.64.131:54403)
0924 01:00:51.636255 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:51.636363 (+ 108us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:51.638650 (+ 2287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:51.639446 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:51.639456 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:51.640006 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0924 01:00:51.640588 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:51.640605 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:51.641776 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:51.641782 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:51.643194 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:51.643207 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:51.644041 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:51.644050 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:51.644302 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:51.645017 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 01:00:51.645039 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0924 01:00:51.647506 (+ 2467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 01:00:51.651126 (+ 3620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:00:51.651135 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:00:51.651153 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:00:51.651606 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:00:51.652009 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:00:51.652014 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:00:51.652019 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:00:51.652217 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:00:51.652771 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 01:00:51.652784 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 01:00:51.653133 (+ 349us) client_negotiation.cc:769] Sending connection context
0924 01:00:51.653432 (+ 299us) client_negotiation.cc:240] Negotiation successful
0924 01:00:51.653973 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":367,"threads_started":1}
I20260924 01:00:51.656695 27199 heartbeater.cc:347] Connected to a master server at 127.8.64.190:40783
I20260924 01:00:51.657016 27199 heartbeater.cc:464] Registering TS with master...
I20260924 01:00:51.657719 27199 heartbeater.cc:511] Master 127.8.64.190:40783 requested a full tablet report, sending...
I20260924 01:00:51.659124 26743 ts_manager.cc:194] Registered new tserver with Master: 47cdf4cc41e74168be0c2f281868d448 (127.8.64.131:41031)
I20260924 01:00:51.659897 26743 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.131@KRBTEST.COM'} at 127.8.64.131:54403
I20260924 01:00:51.661959 8450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 01:00:51.678277 27201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:51.666003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33854 (local address 127.8.64.190:40783)
0924 01:00:51.666230 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:51.666236 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:51.666521 (+ 285us) server_negotiation.cc:407] Connection header received
0924 01:00:51.666644 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:51.666647 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:51.666707 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:51.666788 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:51.668197 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:51.668894 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:51.670196 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:51.670623 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:51.670720 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:51.673043 (+ 2323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:51.673047 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:51.673051 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:00:51.673078 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:51.675609 (+ 2531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:51.676465 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:51.676468 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:51.676471 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:51.676580 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:51.676968 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:51.676971 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:51.676973 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:51.677241 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:51.677430 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:51.677785 (+ 355us) server_negotiation.cc:299] Negotiation successful
0924 01:00:51.677914 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260924 01:00:51.682045 26743 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33854:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 01:00:51.688781 26743 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 01:00:51.715246 27214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:51.701859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:34229 (local address 127.0.0.1:50970)
0924 01:00:51.702892 (+ 1033us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:51.702909 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:51.703030 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:51.704258 (+ 1228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:51.704270 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:51.704310 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 01:00:51.705105 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:51.705131 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:51.708042 (+ 2911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:51.708051 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:51.711646 (+ 3595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:51.711671 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:51.714120 (+ 2449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:51.714129 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:51.714366 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:51.714448 (+ 82us) client_negotiation.cc:769] Sending connection context
0924 01:00:51.714597 (+ 149us) client_negotiation.cc:240] Negotiation successful
0924 01:00:51.714833 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":883,"thread_start_us":731,"threads_started":1}
I20260924 01:00:51.715677 27215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:51.702619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50970 (local address 127.8.64.129:34229)
0924 01:00:51.703264 (+ 645us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:51.703275 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:51.703307 (+ 32us) server_negotiation.cc:407] Connection header received
0924 01:00:51.703421 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:51.703429 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:51.703699 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:51.703929 (+ 230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 01:00:51.705390 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:51.707805 (+ 2415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:51.711931 (+ 4126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:51.713810 (+ 1879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:51.713952 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:51.714236 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:51.715182 (+ 946us) server_negotiation.cc:299] Negotiation successful
0924 01:00:51.715349 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":609,"threads_started":1}
I20260924 01:00:51.719791 26867 tablet_service.cc:1467] Processing CreateTablet for tablet 88304cb0505c416ea3c3d017ea42f11b (DEFAULT_TABLE table=test-table [id=804a5b1650b645aab81c12b04241fc99]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 01:00:51.720603 27212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:51.701341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.131:41031 (local address 127.0.0.1:37156)
0924 01:00:51.702236 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:51.702278 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:51.702485 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:51.713744 (+ 11259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:51.713750 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:51.713783 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 01:00:51.714250 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:51.714265 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:51.716685 (+ 2420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:51.716693 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:51.718506 (+ 1813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:51.718519 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:51.719805 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:51.719811 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:51.719983 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:51.720013 (+ 30us) client_negotiation.cc:769] Sending connection context
0924 01:00:51.720085 (+ 72us) client_negotiation.cc:240] Negotiation successful
0924 01:00:51.720216 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"thread_start_us":516,"threads_started":1}
I20260924 01:00:51.721421 26867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88304cb0505c416ea3c3d017ea42f11b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:00:51.721462 27213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:51.701506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37156 (local address 127.8.64.131:41031)
0924 01:00:51.713045 (+ 11539us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:51.713055 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:51.713105 (+ 50us) server_negotiation.cc:407] Connection header received
0924 01:00:51.713214 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:51.713223 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:51.713499 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:51.714368 (+ 869us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 01:00:51.714706 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:51.716522 (+ 1816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:51.718670 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:51.719575 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:51.719736 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:51.720936 (+ 1200us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:51.721023 (+ 87us) server_negotiation.cc:299] Negotiation successful
0924 01:00:51.721184 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11252,"thread_start_us":11092,"threads_started":1}
I20260924 01:00:51.722909 27217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:51.701859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.130:35025 (local address 127.0.0.1:45382)
0924 01:00:51.713555 (+ 11696us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:51.713575 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:51.713703 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:51.714532 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:51.714536 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:51.714562 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 01:00:51.714940 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:51.714953 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:51.717598 (+ 2645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:51.717607 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:51.719686 (+ 2079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:51.719700 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:51.722050 (+ 2350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:51.722057 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:51.722213 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:51.722244 (+ 31us) client_negotiation.cc:769] Sending connection context
0924 01:00:51.722416 (+ 172us) client_negotiation.cc:240] Negotiation successful
0924 01:00:51.722545 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11580,"mutex_wait_us":1275,"thread_start_us":10213,"threads_started":1}
I20260924 01:00:51.723652 27216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:51.703018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45382 (local address 127.8.64.130:35025)
0924 01:00:51.713310 (+ 10292us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:51.713320 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:51.713828 (+ 508us) server_negotiation.cc:407] Connection header received
0924 01:00:51.714051 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:51.714059 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:51.714329 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:51.714529 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 01:00:51.715157 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:51.717422 (+ 2265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:51.719892 (+ 2470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:51.721604 (+ 1712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:51.721733 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:51.721914 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:51.723127 (+ 1213us) server_negotiation.cc:299] Negotiation successful
0924 01:00:51.723293 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10101,"thread_start_us":9976,"threads_started":1}
I20260924 01:00:51.724735 27134 tablet_service.cc:1467] Processing CreateTablet for tablet 88304cb0505c416ea3c3d017ea42f11b (DEFAULT_TABLE table=test-table [id=804a5b1650b645aab81c12b04241fc99]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 01:00:51.726065 27134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88304cb0505c416ea3c3d017ea42f11b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:00:51.726953 27001 tablet_service.cc:1467] Processing CreateTablet for tablet 88304cb0505c416ea3c3d017ea42f11b (DEFAULT_TABLE table=test-table [id=804a5b1650b645aab81c12b04241fc99]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 01:00:51.728251 27001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88304cb0505c416ea3c3d017ea42f11b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:00:51.737260 27218 tablet_bootstrap.cc:492] T 88304cb0505c416ea3c3d017ea42f11b P 44bffb5db78d4dc79d408b49a9a22c07: Bootstrap starting.
I20260924 01:00:51.739158 27219 tablet_bootstrap.cc:492] T 88304cb0505c416ea3c3d017ea42f11b P 47cdf4cc41e74168be0c2f281868d448: Bootstrap starting.
I20260924 01:00:51.740928 27218 tablet_bootstrap.cc:654] T 88304cb0505c416ea3c3d017ea42f11b P 44bffb5db78d4dc79d408b49a9a22c07: Neither blocks nor log segments found. Creating new log.
I20260924 01:00:51.742095 27218 log.cc:824] T 88304cb0505c416ea3c3d017ea42f11b P 44bffb5db78d4dc79d408b49a9a22c07: Log is configured to *not* fsync() on all Append() calls
I20260924 01:00:51.742458 27220 tablet_bootstrap.cc:492] T 88304cb0505c416ea3c3d017ea42f11b P cd884c3bb146494594cbb14d53cf5a73: Bootstrap starting.
I20260924 01:00:51.742983 27219 tablet_bootstrap.cc:654] T 88304cb0505c416ea3c3d017ea42f11b P 47cdf4cc41e74168be0c2f281868d448: Neither blocks nor log segments found. Creating new log.
I20260924 01:00:51.744261 27219 log.cc:824] T 88304cb0505c416ea3c3d017ea42f11b P 47cdf4cc41e74168be0c2f281868d448: Log is configured to *not* fsync() on all Append() calls
I20260924 01:00:51.746188 27220 tablet_bootstrap.cc:654] T 88304cb0505c416ea3c3d017ea42f11b P cd884c3bb146494594cbb14d53cf5a73: Neither blocks nor log segments found. Creating new log.
I20260924 01:00:51.746325 27218 tablet_bootstrap.cc:492] T 88304cb0505c416ea3c3d017ea42f11b P 44bffb5db78d4dc79d408b49a9a22c07: No bootstrap required, opened a new log
I20260924 01:00:51.746618 27218 ts_tablet_manager.cc:1397] T 88304cb0505c416ea3c3d017ea42f11b P 44bffb5db78d4dc79d408b49a9a22c07: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.002s
I20260924 01:00:51.747220 27220 log.cc:824] T 88304cb0505c416ea3c3d017ea42f11b P cd884c3bb146494594cbb14d53cf5a73: Log is configured to *not* fsync() on all Append() calls
I20260924 01:00:51.747325 27219 tablet_bootstrap.cc:492] T 88304cb0505c416ea3c3d017ea42f11b P 47cdf4cc41e74168be0c2f281868d448: No bootstrap required, opened a new log
I20260924 01:00:51.747567 27219 ts_tablet_manager.cc:1397] T 88304cb0505c416ea3c3d017ea42f11b P 47cdf4cc41e74168be0c2f281868d448: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260924 01:00:51.749706 27220 tablet_bootstrap.cc:492] T 88304cb0505c416ea3c3d017ea42f11b P cd884c3bb146494594cbb14d53cf5a73: No bootstrap required, opened a new log
I20260924 01:00:51.749966 27220 ts_tablet_manager.cc:1397] T 88304cb0505c416ea3c3d017ea42f11b P cd884c3bb146494594cbb14d53cf5a73: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260924 01:00:51.757587 27220 raft_consensus.cc:348] T 88304cb0505c416ea3c3d017ea42f11b P cd884c3bb146494594cbb14d53cf5a73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47cdf4cc41e74168be0c2f281868d448" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 41031 } } peers { permanent_uuid: "44bffb5db78d4dc79d408b49a9a22c07" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 34229 } } peers { permanent_uuid: "cd884c3bb146494594cbb14d53cf5a73" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 35025 } }
I20260924 01:00:51.757607 27218 raft_consensus.cc:348] T 88304cb0505c416ea3c3d017ea42f11b P 44bffb5db78d4dc79d408b49a9a22c07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47cdf4cc41e74168be0c2f281868d448" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 41031 } } peers { permanent_uuid: "44bffb5db78d4dc79d408b49a9a22c07" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 34229 } } peers { permanent_uuid: "cd884c3bb146494594cbb14d53cf5a73" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 35025 } }
I20260924 01:00:51.757967 27220 raft_consensus.cc:374] T 88304cb0505c416ea3c3d017ea42f11b P cd884c3bb146494594cbb14d53cf5a73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 01:00:51.758013 27218 raft_consensus.cc:374] T 88304cb0505c416ea3c3d017ea42f11b P 44bffb5db78d4dc79d408b49a9a22c07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 01:00:51.758059 27220 raft_consensus.cc:729] T 88304cb0505c416ea3c3d017ea42f11b P cd884c3bb146494594cbb14d53cf5a73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd884c3bb146494594cbb14d53cf5a73, State: Initialized, Role: FOLLOWER
I20260924 01:00:51.758124 27218 raft_consensus.cc:729] T 88304cb0505c416ea3c3d017ea42f11b P 44bffb5db78d4dc79d408b49a9a22c07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44bffb5db78d4dc79d408b49a9a22c07, State: Initialized, Role: FOLLOWER
I20260924 01:00:51.758358 27219 raft_consensus.cc:348] T 88304cb0505c416ea3c3d017ea42f11b P 47cdf4cc41e74168be0c2f281868d448 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47cdf4cc41e74168be0c2f281868d448" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 41031 } } peers { permanent_uuid: "44bffb5db78d4dc79d408b49a9a22c07" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 34229 } } peers { permanent_uuid: "cd884c3bb146494594cbb14d53cf5a73" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 35025 } }
I20260924 01:00:51.758771 27219 raft_consensus.cc:374] T 88304cb0505c416ea3c3d017ea42f11b P 47cdf4cc41e74168be0c2f281868d448 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 01:00:51.758876 27219 raft_consensus.cc:729] T 88304cb0505c416ea3c3d017ea42f11b P 47cdf4cc41e74168be0c2f281868d448 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47cdf4cc41e74168be0c2f281868d448, State: Initialized, Role: FOLLOWER
I20260924 01:00:51.758785 27218 consensus_queue.cc:260] T 88304cb0505c416ea3c3d017ea42f11b P 44bffb5db78d4dc79d408b49a9a22c07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47cdf4cc41e74168be0c2f281868d448" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 41031 } } peers { permanent_uuid: "44bffb5db78d4dc79d408b49a9a22c07" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 34229 } } peers { permanent_uuid: "cd884c3bb146494594cbb14d53cf5a73" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 35025 } }
I20260924 01:00:51.758766 27220 consensus_queue.cc:260] T 88304cb0505c416ea3c3d017ea42f11b P cd884c3bb146494594cbb14d53cf5a73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47cdf4cc41e74168be0c2f281868d448" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 41031 } } peers { permanent_uuid: "44bffb5db78d4dc79d408b49a9a22c07" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 34229 } } peers { permanent_uuid: "cd884c3bb146494594cbb14d53cf5a73" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 35025 } }
I20260924 01:00:51.759564 27219 consensus_queue.cc:260] T 88304cb0505c416ea3c3d017ea42f11b P 47cdf4cc41e74168be0c2f281868d448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47cdf4cc41e74168be0c2f281868d448" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 41031 } } peers { permanent_uuid: "44bffb5db78d4dc79d408b49a9a22c07" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 34229 } } peers { permanent_uuid: "cd884c3bb146494594cbb14d53cf5a73" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 35025 } }
I20260924 01:00:51.760993 27199 heartbeater.cc:503] Master 127.8.64.190:40783 was elected leader, sending a full tablet report...
I20260924 01:00:51.761886 27219 ts_tablet_manager.cc:1428] T 88304cb0505c416ea3c3d017ea42f11b P 47cdf4cc41e74168be0c2f281868d448: Time spent starting tablet: real 0.014s user 0.012s sys 0.002s
I20260924 01:00:51.763916 27220 ts_tablet_manager.cc:1428] T 88304cb0505c416ea3c3d017ea42f11b P cd884c3bb146494594cbb14d53cf5a73: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260924 01:00:51.764423 27066 heartbeater.cc:503] Master 127.8.64.190:40783 was elected leader, sending a full tablet report...
I20260924 01:00:51.765781 27218 ts_tablet_manager.cc:1428] T 88304cb0505c416ea3c3d017ea42f11b P 44bffb5db78d4dc79d408b49a9a22c07: Time spent starting tablet: real 0.019s user 0.016s sys 0.002s
W20260924 01:00:51.856577 26933 tablet.cc:2369] T 88304cb0505c416ea3c3d017ea42f11b P 44bffb5db78d4dc79d408b49a9a22c07: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 01:00:51.861403 27224 raft_consensus.cc:482] T 88304cb0505c416ea3c3d017ea42f11b P 44bffb5db78d4dc79d408b49a9a22c07 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 01:00:51.861749 27224 raft_consensus.cc:504] T 88304cb0505c416ea3c3d017ea42f11b P 44bffb5db78d4dc79d408b49a9a22c07 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47cdf4cc41e74168be0c2f281868d448" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 41031 } } peers { permanent_uuid: "44bffb5db78d4dc79d408b49a9a22c07" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 34229 } } peers { permanent_uuid: "cd884c3bb146494594cbb14d53cf5a73" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 35025 } }
I20260924 01:00:51.863533 27224 leader_election.cc:288] T 88304cb0505c416ea3c3d017ea42f11b P 44bffb5db78d4dc79d408b49a9a22c07 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 47cdf4cc41e74168be0c2f281868d448 (127.8.64.131:41031), cd884c3bb146494594cbb14d53cf5a73 (127.8.64.130:35025)
W20260924 01:00:51.865926 27067 tablet.cc:2369] T 88304cb0505c416ea3c3d017ea42f11b P cd884c3bb146494594cbb14d53cf5a73: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 01:00:51.871112 27213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:51.863839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:38065 (local address 127.8.64.131:41031)
0924 01:00:51.864073 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:51.864083 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:51.864940 (+ 857us) server_negotiation.cc:407] Connection header received
0924 01:00:51.865168 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:51.865180 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:51.865239 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:51.865338 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 01:00:51.866229 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:51.867247 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:51.869125 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:51.870090 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:51.870207 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:51.870258 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:51.870697 (+ 439us) server_negotiation.cc:299] Negotiation successful
0924 01:00:51.870841 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260924 01:00:51.871166 27227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:51.863688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.131:41031 (local address 127.8.64.129:38065)
0924 01:00:51.864831 (+ 1143us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:51.864847 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:51.865026 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:51.865357 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:51.865361 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:51.865386 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 01:00:51.866049 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:51.866064 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:51.867361 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:51.867367 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:51.868962 (+ 1595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:51.868975 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:51.870322 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:51.870330 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:51.870505 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:51.870541 (+ 36us) client_negotiation.cc:769] Sending connection context
0924 01:00:51.870648 (+ 107us) client_negotiation.cc:240] Negotiation successful
0924 01:00:51.870791 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1032,"mutex_wait_us":24,"thread_start_us":655,"threads_started":1}
I20260924 01:00:51.871747 27216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:51.864140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:53637 (local address 127.8.64.130:35025)
0924 01:00:51.864329 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:51.864337 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:51.865048 (+ 711us) server_negotiation.cc:407] Connection header received
0924 01:00:51.865238 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:51.865244 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:51.865309 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:51.865401 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 01:00:51.866420 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:51.867459 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:51.869660 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:51.870692 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:51.870831 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:51.870901 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:51.871318 (+ 417us) server_negotiation.cc:299] Negotiation successful
0924 01:00:51.871514 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260924 01:00:51.872283 27228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:51.863843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.130:35025 (local address 127.8.64.129:53637)
0924 01:00:51.864944 (+ 1101us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:51.864960 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:51.865110 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:51.865414 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:51.865418 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:51.865473 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 01:00:51.866205 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:51.866219 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:51.867600 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:51.867604 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:51.869434 (+ 1830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:51.869449 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:51.870921 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:51.870928 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:51.871105 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:51.871152 (+ 47us) client_negotiation.cc:769] Sending connection context
0924 01:00:51.871924 (+ 772us) client_negotiation.cc:240] Negotiation successful
0924 01:00:51.872032 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1004,"mutex_wait_us":41,"thread_start_us":615,"threads_started":1}
I20260924 01:00:51.872171 27154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88304cb0505c416ea3c3d017ea42f11b" candidate_uuid: "44bffb5db78d4dc79d408b49a9a22c07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47cdf4cc41e74168be0c2f281868d448" is_pre_election: true
I20260924 01:00:51.872685 27154 raft_consensus.cc:2454] T 88304cb0505c416ea3c3d017ea42f11b P 47cdf4cc41e74168be0c2f281868d448 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44bffb5db78d4dc79d408b49a9a22c07 in term 0.
I20260924 01:00:51.873291 26822 leader_election.cc:302] T 88304cb0505c416ea3c3d017ea42f11b P 44bffb5db78d4dc79d408b49a9a22c07 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44bffb5db78d4dc79d408b49a9a22c07, 47cdf4cc41e74168be0c2f281868d448; no voters:
I20260924 01:00:51.873257 27021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88304cb0505c416ea3c3d017ea42f11b" candidate_uuid: "44bffb5db78d4dc79d408b49a9a22c07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd884c3bb146494594cbb14d53cf5a73" is_pre_election: true
I20260924 01:00:51.873678 27224 raft_consensus.cc:2781] T 88304cb0505c416ea3c3d017ea42f11b P 44bffb5db78d4dc79d408b49a9a22c07 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 01:00:51.873700 27021 raft_consensus.cc:2454] T 88304cb0505c416ea3c3d017ea42f11b P cd884c3bb146494594cbb14d53cf5a73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44bffb5db78d4dc79d408b49a9a22c07 in term 0.
I20260924 01:00:51.873790 27224 raft_consensus.cc:482] T 88304cb0505c416ea3c3d017ea42f11b P 44bffb5db78d4dc79d408b49a9a22c07 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 01:00:51.873922 27224 raft_consensus.cc:3037] T 88304cb0505c416ea3c3d017ea42f11b P 44bffb5db78d4dc79d408b49a9a22c07 [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:00:51.876230 27224 raft_consensus.cc:504] T 88304cb0505c416ea3c3d017ea42f11b P 44bffb5db78d4dc79d408b49a9a22c07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47cdf4cc41e74168be0c2f281868d448" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 41031 } } peers { permanent_uuid: "44bffb5db78d4dc79d408b49a9a22c07" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 34229 } } peers { permanent_uuid: "cd884c3bb146494594cbb14d53cf5a73" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 35025 } }
I20260924 01:00:51.876924 27224 leader_election.cc:288] T 88304cb0505c416ea3c3d017ea42f11b P 44bffb5db78d4dc79d408b49a9a22c07 [CANDIDATE]: Term 1 election: Requested vote from peers 47cdf4cc41e74168be0c2f281868d448 (127.8.64.131:41031), cd884c3bb146494594cbb14d53cf5a73 (127.8.64.130:35025)
I20260924 01:00:51.877321 27154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88304cb0505c416ea3c3d017ea42f11b" candidate_uuid: "44bffb5db78d4dc79d408b49a9a22c07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47cdf4cc41e74168be0c2f281868d448"
I20260924 01:00:51.877540 27154 raft_consensus.cc:3037] T 88304cb0505c416ea3c3d017ea42f11b P 47cdf4cc41e74168be0c2f281868d448 [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:00:51.877489 27021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88304cb0505c416ea3c3d017ea42f11b" candidate_uuid: "44bffb5db78d4dc79d408b49a9a22c07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd884c3bb146494594cbb14d53cf5a73"
I20260924 01:00:51.877707 27021 raft_consensus.cc:3037] T 88304cb0505c416ea3c3d017ea42f11b P cd884c3bb146494594cbb14d53cf5a73 [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:00:51.879813 27154 raft_consensus.cc:2454] T 88304cb0505c416ea3c3d017ea42f11b P 47cdf4cc41e74168be0c2f281868d448 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44bffb5db78d4dc79d408b49a9a22c07 in term 1.
I20260924 01:00:51.879894 27021 raft_consensus.cc:2454] T 88304cb0505c416ea3c3d017ea42f11b P cd884c3bb146494594cbb14d53cf5a73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44bffb5db78d4dc79d408b49a9a22c07 in term 1.
I20260924 01:00:51.880270 26822 leader_election.cc:302] T 88304cb0505c416ea3c3d017ea42f11b P 44bffb5db78d4dc79d408b49a9a22c07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44bffb5db78d4dc79d408b49a9a22c07, 47cdf4cc41e74168be0c2f281868d448; no voters:
I20260924 01:00:51.880595 27224 raft_consensus.cc:2781] T 88304cb0505c416ea3c3d017ea42f11b P 44bffb5db78d4dc79d408b49a9a22c07 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 01:00:51.881062 27224 raft_consensus.cc:686] T 88304cb0505c416ea3c3d017ea42f11b P 44bffb5db78d4dc79d408b49a9a22c07 [term 1 LEADER]: Becoming Leader. State: Replica: 44bffb5db78d4dc79d408b49a9a22c07, State: Running, Role: LEADER
I20260924 01:00:51.881536 27224 consensus_queue.cc:237] T 88304cb0505c416ea3c3d017ea42f11b P 44bffb5db78d4dc79d408b49a9a22c07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47cdf4cc41e74168be0c2f281868d448" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 41031 } } peers { permanent_uuid: "44bffb5db78d4dc79d408b49a9a22c07" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 34229 } } peers { permanent_uuid: "cd884c3bb146494594cbb14d53cf5a73" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 35025 } }
W20260924 01:00:51.883252 27200 tablet.cc:2369] T 88304cb0505c416ea3c3d017ea42f11b P 47cdf4cc41e74168be0c2f281868d448: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 01:00:51.886075 26743 catalog_manager.cc:5495] T 88304cb0505c416ea3c3d017ea42f11b P 44bffb5db78d4dc79d408b49a9a22c07 reported cstate change: term changed from 0 to 1, leader changed from <none> to 44bffb5db78d4dc79d408b49a9a22c07 (127.8.64.129). New cstate: current_term: 1 leader_uuid: "44bffb5db78d4dc79d408b49a9a22c07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47cdf4cc41e74168be0c2f281868d448" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 41031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "44bffb5db78d4dc79d408b49a9a22c07" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 34229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd884c3bb146494594cbb14d53cf5a73" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 35025 } health_report { overall_health: UNKNOWN } } }
I20260924 01:00:51.916498 27215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:51.910893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50982 (local address 127.8.64.129:34229)
0924 01:00:51.911076 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:51.911081 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:51.911099 (+ 18us) server_negotiation.cc:407] Connection header received
0924 01:00:51.911252 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:51.911256 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:51.911332 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:51.911453 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 01:00:51.911867 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:51.912574 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:51.914001 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:51.914516 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:51.914619 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:51.915205 (+ 586us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 01:00:51.915744 (+ 539us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 01:00:51.915882 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:51.916230 (+ 348us) server_negotiation.cc:299] Negotiation successful
0924 01:00:51.916270 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260924 01:00:51.924809 27021 raft_consensus.cc:1261] T 88304cb0505c416ea3c3d017ea42f11b P cd884c3bb146494594cbb14d53cf5a73 [term 1 FOLLOWER]: Refusing update from remote peer 44bffb5db78d4dc79d408b49a9a22c07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 01:00:51.924811 27154 raft_consensus.cc:1261] T 88304cb0505c416ea3c3d017ea42f11b P 47cdf4cc41e74168be0c2f281868d448 [term 1 FOLLOWER]: Refusing update from remote peer 44bffb5db78d4dc79d408b49a9a22c07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 01:00:51.925802 27224 consensus_queue.cc:1035] T 88304cb0505c416ea3c3d017ea42f11b P 44bffb5db78d4dc79d408b49a9a22c07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd884c3bb146494594cbb14d53cf5a73" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 35025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 01:00:51.926100 27229 consensus_queue.cc:1035] T 88304cb0505c416ea3c3d017ea42f11b P 44bffb5db78d4dc79d408b49a9a22c07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47cdf4cc41e74168be0c2f281868d448" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 41031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 01:00:51.963867 26742 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33854:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 01:00:51.964182 26742 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33854:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 01:00:51.967968 26742 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 174a27e379ae451684f8f51ec3d86778: Sending DeleteTablet for 3 replicas of tablet 88304cb0505c416ea3c3d017ea42f11b
I20260924 01:00:51.969218 26867 tablet_service.cc:1514] Processing DeleteTablet for tablet 88304cb0505c416ea3c3d017ea42f11b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 01:00:51 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:50970
I20260924 01:00:51.969209 27134 tablet_service.cc:1514] Processing DeleteTablet for tablet 88304cb0505c416ea3c3d017ea42f11b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 01:00:51 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:37156
I20260924 01:00:51.969316 27001 tablet_service.cc:1514] Processing DeleteTablet for tablet 88304cb0505c416ea3c3d017ea42f11b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 01:00:51 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:45382
I20260924 01:00:51.975114 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 26804
I20260924 01:00:51.991081 27244 tablet_replica.cc:330] T 88304cb0505c416ea3c3d017ea42f11b P cd884c3bb146494594cbb14d53cf5a73: stopping tablet replica
I20260924 01:00:51.991338 27242 tablet_replica.cc:330] T 88304cb0505c416ea3c3d017ea42f11b P 47cdf4cc41e74168be0c2f281868d448: stopping tablet replica
I20260924 01:00:51.991783 27244 raft_consensus.cc:2229] T 88304cb0505c416ea3c3d017ea42f11b P cd884c3bb146494594cbb14d53cf5a73 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 01:00:51.991781 27242 raft_consensus.cc:2229] T 88304cb0505c416ea3c3d017ea42f11b P 47cdf4cc41e74168be0c2f281868d448 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 01:00:51.992329 27242 raft_consensus.cc:2258] T 88304cb0505c416ea3c3d017ea42f11b P 47cdf4cc41e74168be0c2f281868d448 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 01:00:51.992329 27244 raft_consensus.cc:2258] T 88304cb0505c416ea3c3d017ea42f11b P cd884c3bb146494594cbb14d53cf5a73 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 01:00:51.994702 27244 ts_tablet_manager.cc:1905] T 88304cb0505c416ea3c3d017ea42f11b P cd884c3bb146494594cbb14d53cf5a73: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 01:00:51.996186 27242 ts_tablet_manager.cc:1905] T 88304cb0505c416ea3c3d017ea42f11b P 47cdf4cc41e74168be0c2f281868d448: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 01:00:52.001300 27243 tablet_replica.cc:330] T 88304cb0505c416ea3c3d017ea42f11b P 44bffb5db78d4dc79d408b49a9a22c07: stopping tablet replica
Sep 24 01:00:51 dist-test-slave-mmhs krb5kdc[26698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.129@KRBTEST.COM
I20260924 01:00:52.001761 27243 raft_consensus.cc:2229] T 88304cb0505c416ea3c3d017ea42f11b P 44bffb5db78d4dc79d408b49a9a22c07 [term 1 LEADER]: Raft consensus shutting down.
I20260924 01:00:52.002856 27244 ts_tablet_manager.cc:1918] T 88304cb0505c416ea3c3d017ea42f11b P cd884c3bb146494594cbb14d53cf5a73: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 01:00:52.003074 27244 log.cc:1196] T 88304cb0505c416ea3c3d017ea42f11b P cd884c3bb146494594cbb14d53cf5a73: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-1/wal/wals/88304cb0505c416ea3c3d017ea42f11b
I20260924 01:00:52.003700 27244 ts_tablet_manager.cc:1939] T 88304cb0505c416ea3c3d017ea42f11b P cd884c3bb146494594cbb14d53cf5a73: Deleting consensus metadata
I20260924 01:00:52.005815 26728 catalog_manager.cc:4841] TS cd884c3bb146494594cbb14d53cf5a73 (127.8.64.130:35025): tablet 88304cb0505c416ea3c3d017ea42f11b (table test-table [id=804a5b1650b645aab81c12b04241fc99]) successfully deleted
I20260924 01:00:52.007145 27242 ts_tablet_manager.cc:1918] T 88304cb0505c416ea3c3d017ea42f11b P 47cdf4cc41e74168be0c2f281868d448: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 01:00:52.007313 27242 log.cc:1196] T 88304cb0505c416ea3c3d017ea42f11b P 47cdf4cc41e74168be0c2f281868d448: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790211343099518-8450-0/minicluster-data/ts-2/wal/wals/88304cb0505c416ea3c3d017ea42f11b
I20260924 01:00:52.007849 27242 ts_tablet_manager.cc:1939] T 88304cb0505c416ea3c3d017ea42f11b P 47cdf4cc41e74168be0c2f281868d448: Deleting consensus metadata
I20260924 01:00:52.009791 26729 catalog_manager.cc:4841] TS 47cdf4cc41e74168be0c2f281868d448 (127.8.64.131:41031): tablet 88304cb0505c416ea3c3d017ea42f11b (table test-table [id=804a5b1650b645aab81c12b04241fc99]) successfully deleted
I20260924 01:00:52.009793 27243 raft_consensus.cc:2258] T 88304cb0505c416ea3c3d017ea42f11b P 44bffb5db78d4dc79d408b49a9a22c07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 01:00:52.012372 27243 ts_tablet_manager.cc:1905] T 88304cb0505c416ea3c3d017ea42f11b P 44bffb5db78d4dc79d408b49a9a22c07: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 01:00:52.013657 27215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:51.976436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50998 (local address 127.8.64.129:34229)
0924 01:00:51.976647 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:51.976653 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:51.990351 (+ 13698us) server_negotiation.cc:407] Connection header received
0924 01:00:51.990941 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:51.990948 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:51.991043 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:51.991151 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:51.993113 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:51.994376 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:51.996311 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:51.997029 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:51.997129 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:52.002695 (+ 5566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:52.002711 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:52.002729 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:00:52.002777 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:52.006862 (+ 4085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:52.010771 (+ 3909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:52.010781 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:52.010785 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:52.010959 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:52.011459 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:52.011465 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:52.011469 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:52.011905 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:52.012254 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:52.013022 (+ 768us) server_negotiation.cc:299] Negotiation successful
0924 01:00:52.013266 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260924 01:00:52.014688 26927 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 01:00:52.119122 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 26804
W20260924 01:00:52.139304 26727 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.64.129:34229 (error 108)
I20260924 01:00:52.139640 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 26938
I20260924 01:00:52.141265 27217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:52.140838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:34229 (local address 127.0.0.1:51006)
0924 01:00:52.141010 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:52.141111 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.129:34229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260924 01:00:52.141606 26727 catalog_manager.cc:4584] TS 44bffb5db78d4dc79d408b49a9a22c07 (127.8.64.129:34229): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 88304cb0505c416ea3c3d017ea42f11b: Network error: Client connection negotiation failed: client connection to 127.8.64.129:34229: connect: Connection refused (error 111)
Sep 24 01:00:52 dist-test-slave-mmhs krb5kdc[26698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.130@KRBTEST.COM
I20260924 01:00:52.155583 27216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:52.140532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45384 (local address 127.8.64.130:35025)
0924 01:00:52.140742 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:52.140749 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:52.141094 (+ 345us) server_negotiation.cc:407] Connection header received
0924 01:00:52.141320 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:52.141326 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:52.141433 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:52.141526 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:52.143059 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:52.143794 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:52.145085 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:52.145469 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:52.145572 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:52.149873 (+ 4301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:52.149884 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:52.149899 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:00:52.149934 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:52.152934 (+ 3000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:52.153664 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:52.153671 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:52.153673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:52.153790 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:52.154128 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:52.154131 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:52.154133 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:52.154610 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:52.154837 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:52.155100 (+ 263us) server_negotiation.cc:299] Negotiation successful
0924 01:00:52.155256 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260924 01:00:52.156288 27061 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 01:00:52.195384 27217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:52.194900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:34229 (local address 127.0.0.1:51016)
0924 01:00:52.195137 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:52.195224 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.129:34229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260924 01:00:52.196425 27217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:52.196071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:34229 (local address 127.0.0.1:51024)
0924 01:00:52.196237 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:52.196315 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.129:34229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260924 01:00:52.245146 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 26938
I20260924 01:00:52.256937 27217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:52.256467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:34229 (local address 127.0.0.1:51030)
0924 01:00:52.256699 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:52.256788 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.129:34229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260924 01:00:52.257970 27217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:52.257656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:34229 (local address 127.0.0.1:51040)
0924 01:00:52.257794 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:52.257860 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.129:34229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260924 01:00:52.263599 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 27071
Sep 24 01:00:52 dist-test-slave-mmhs krb5kdc[26698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.131@KRBTEST.COM
I20260924 01:00:52.279423 27213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:52.264424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37172 (local address 127.8.64.131:41031)
0924 01:00:52.264606 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:52.264610 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:52.264628 (+ 18us) server_negotiation.cc:407] Connection header received
0924 01:00:52.264687 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:52.264691 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:52.264756 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:52.264847 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:52.266273 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:52.267154 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:52.268545 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:52.268970 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:52.269076 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:52.273638 (+ 4562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:52.273652 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:52.273668 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:00:52.273701 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:52.276852 (+ 3151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:52.277548 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:52.277556 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:52.277558 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:52.277696 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:52.278065 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:52.278068 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:52.278070 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:52.278484 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:52.278720 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:52.279000 (+ 280us) server_negotiation.cc:299] Negotiation successful
0924 01:00:52.279169 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260924 01:00:52.280051 27194 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 01:00:52.339058 27217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:52.338439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:34229 (local address 127.0.0.1:51046)
0924 01:00:52.338711 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:52.338829 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.129:34229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260924 01:00:52.340273 27217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:52.339860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:34229 (local address 127.0.0.1:51050)
0924 01:00:52.340080 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:52.340151 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.129:34229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260924 01:00:52.369249 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 27071
I20260924 01:00:52.386379 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 26711
I20260924 01:00:52.387475 26773 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 01:00:52.498858 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 26711
2026-09-24T01:00:52Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4282 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-24T01:00:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T01:00:52Z Disabled control of system clock
I20260924 01:00:52.579813 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:42981
--webserver_interface=127.8.64.190
--webserver_port=0
--builtin_ntp_servers=127.8.64.148:45671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:42981
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 01:00:52.904276 27254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:00:52.904548 27254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:00:52.904628 27254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:00:52.911679 27254 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 01:00:52.911782 27254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:00:52.911819 27254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:00:52.911878 27254 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 01:00:52.911921 27254 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 01:00:52.921001 27254 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:45671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:42981
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:42981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:00:52.922227 27254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:00:52.924093 27254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:00:52.935976 27259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:52.936025 27260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:52.935912 27262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:00:52.937580 27254 server_base.cc:1034] running on GCE node
I20260924 01:00:52.938387 27254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:00:52.939752 27254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:00:52.941004 27254 hybrid_clock.cc:648] HybridClock initialized: now 1790211652940938 us; error 62 us; skew 500 ppm
I20260924 01:00:52.941375 27254 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 01:00:52.947817 27254 webserver.cc:460] Webserver started at http://127.8.64.190:44017/ using document root <none> and password file <none>
I20260924 01:00:52.948527 27254 fs_manager.cc:359] Metadata directory not provided
I20260924 01:00:52.948665 27254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:00:52.948997 27254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:00:52.951930 27254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "2ff8d2ed07864ed580faf26b100e581a"
format_stamp: "Formatted at 2026-09-24 01:00:52 on dist-test-slave-mmhs"
server_key: "eaf36e3c1f9cdb7656ce4e412321454c"
server_key_iv: "bf56542aaa9c8458ddf30434d7c37270"
server_key_version: "encryptionkey@0"
I20260924 01:00:52.952721 27254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "2ff8d2ed07864ed580faf26b100e581a"
format_stamp: "Formatted at 2026-09-24 01:00:52 on dist-test-slave-mmhs"
server_key: "eaf36e3c1f9cdb7656ce4e412321454c"
server_key_iv: "bf56542aaa9c8458ddf30434d7c37270"
server_key_version: "encryptionkey@0"
I20260924 01:00:52.957651 27254 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260924 01:00:52.960940 27268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:52.962211 27254 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260924 01:00:52.962440 27254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "2ff8d2ed07864ed580faf26b100e581a"
format_stamp: "Formatted at 2026-09-24 01:00:52 on dist-test-slave-mmhs"
server_key: "eaf36e3c1f9cdb7656ce4e412321454c"
server_key_iv: "bf56542aaa9c8458ddf30434d7c37270"
server_key_version: "encryptionkey@0"
I20260924 01:00:52.962651 27254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:00:52.981312 27254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:00:52.982115 27254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:00:52.982424 27254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:00:53.001855 27254 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.190:42981
I20260924 01:00:53.001878 27319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.190:42981 every 8 connection(s)
I20260924 01:00:53.003233 27254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 01:00:53.003955 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 27254
I20260924 01:00:53.004142 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 01:00:53.004658 8450 external_mini_cluster.cc:1428] Setting key c0d9441635b6f15c7ce4646b090b6f66
I20260924 01:00:53.007804 27320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:00:53.020597 27323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:53.006545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49020 (local address 127.8.64.190:42981)
0924 01:00:53.007610 (+ 1065us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:53.007623 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:53.007671 (+ 48us) server_negotiation.cc:407] Connection header received
0924 01:00:53.010159 (+ 2488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:53.010186 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:53.010785 (+ 599us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:53.011246 (+ 461us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:53.011776 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:53.013848 (+ 2072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:53.015363 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:53.016391 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:53.016573 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:53.017734 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:53.017744 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:53.017766 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:00:53.018496 (+ 730us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:53.018595 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:00:53.018624 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:53.018764 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:53.018997 (+ 233us) server_negotiation.cc:299] Negotiation successful
0924 01:00:53.019343 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":337,"threads_started":1}
I20260924 01:00:53.021857 27320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ff8d2ed07864ed580faf26b100e581a: Bootstrap starting.
I20260924 01:00:53.025005 27320 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ff8d2ed07864ed580faf26b100e581a: Neither blocks nor log segments found. Creating new log.
I20260924 01:00:53.025982 27320 log.cc:824] T 00000000000000000000000000000000 P 2ff8d2ed07864ed580faf26b100e581a: Log is configured to *not* fsync() on all Append() calls
I20260924 01:00:53.028791 27320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ff8d2ed07864ed580faf26b100e581a: No bootstrap required, opened a new log
I20260924 01:00:53.036062 27320 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ff8d2ed07864ed580faf26b100e581a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ff8d2ed07864ed580faf26b100e581a" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42981 } }
I20260924 01:00:53.036399 27320 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ff8d2ed07864ed580faf26b100e581a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 01:00:53.036473 27320 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ff8d2ed07864ed580faf26b100e581a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ff8d2ed07864ed580faf26b100e581a, State: Initialized, Role: FOLLOWER
I20260924 01:00:53.036932 27320 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ff8d2ed07864ed580faf26b100e581a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ff8d2ed07864ed580faf26b100e581a" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42981 } }
I20260924 01:00:53.037185 27320 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ff8d2ed07864ed580faf26b100e581a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 01:00:53.037288 27320 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ff8d2ed07864ed580faf26b100e581a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 01:00:53.037411 27320 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ff8d2ed07864ed580faf26b100e581a [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:00:53.039923 27320 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ff8d2ed07864ed580faf26b100e581a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ff8d2ed07864ed580faf26b100e581a" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42981 } }
I20260924 01:00:53.040407 27320 leader_election.cc:302] T 00000000000000000000000000000000 P 2ff8d2ed07864ed580faf26b100e581a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ff8d2ed07864ed580faf26b100e581a; no voters:
I20260924 01:00:53.040922 27320 leader_election.cc:288] T 00000000000000000000000000000000 P 2ff8d2ed07864ed580faf26b100e581a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 01:00:53.041087 27325 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ff8d2ed07864ed580faf26b100e581a [term 1 FOLLOWER]: Leader election won for term 1
I20260924 01:00:53.041805 27325 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ff8d2ed07864ed580faf26b100e581a [term 1 LEADER]: Becoming Leader. State: Replica: 2ff8d2ed07864ed580faf26b100e581a, State: Running, Role: LEADER
I20260924 01:00:53.042367 27325 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ff8d2ed07864ed580faf26b100e581a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ff8d2ed07864ed580faf26b100e581a" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42981 } }
I20260924 01:00:53.043002 27320 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ff8d2ed07864ed580faf26b100e581a [sys.catalog]: configured and running, proceeding with master startup.
I20260924 01:00:53.045488 27326 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ff8d2ed07864ed580faf26b100e581a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ff8d2ed07864ed580faf26b100e581a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ff8d2ed07864ed580faf26b100e581a" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42981 } } }
I20260924 01:00:53.045948 27326 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ff8d2ed07864ed580faf26b100e581a [sys.catalog]: This master's current role is: LEADER
I20260924 01:00:53.045490 27327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ff8d2ed07864ed580faf26b100e581a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ff8d2ed07864ed580faf26b100e581a. Latest consensus state: current_term: 1 leader_uuid: "2ff8d2ed07864ed580faf26b100e581a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ff8d2ed07864ed580faf26b100e581a" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 42981 } } }
I20260924 01:00:53.047340 27327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ff8d2ed07864ed580faf26b100e581a [sys.catalog]: This master's current role is: LEADER
I20260924 01:00:53.047482 27333 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 01:00:53.053901 27333 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 01:00:53.062135 27333 catalog_manager.cc:1347] Generated new cluster ID: 2c0600a2551f40cdae5331572b55d872
I20260924 01:00:53.062242 27333 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 01:00:53.085386 27333 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 01:00:53.086747 27333 catalog_manager.cc:1504] Loading token signing keys...
I20260924 01:00:53.095468 27333 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ff8d2ed07864ed580faf26b100e581a: Generated new TSK 0
I20260924 01:00:53.096336 27333 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 01:00:53.104975 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.129:0
--local_ip_for_outbound_sockets=127.8.64.129
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:42981
--builtin_ntp_servers=127.8.64.148:45671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 01:00:53.466457 27344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:00:53.466868 27344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:00:53.467044 27344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:00:53.475461 27344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:00:53.475605 27344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:00:53.475769 27344 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.129
I20260924 01:00:53.485131 27344 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:45671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:42981
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.129
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:00:53.486675 27344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:00:53.488608 27344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:00:53.501966 27350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:53.501983 27349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:53.502023 27352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:00:53.502780 27344 server_base.cc:1034] running on GCE node
I20260924 01:00:53.503396 27344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:00:53.504380 27344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:00:53.505586 27344 hybrid_clock.cc:648] HybridClock initialized: now 1790211653505539 us; error 35 us; skew 500 ppm
I20260924 01:00:53.505911 27344 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 01:00:53.508284 27344 webserver.cc:460] Webserver started at http://127.8.64.129:32863/ using document root <none> and password file <none>
I20260924 01:00:53.508998 27344 fs_manager.cc:359] Metadata directory not provided
I20260924 01:00:53.509087 27344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:00:53.509449 27344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:00:53.512408 27344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-0/data/instance:
uuid: "205155e209e3469eb9ff04215bd1c0c2"
format_stamp: "Formatted at 2026-09-24 01:00:53 on dist-test-slave-mmhs"
server_key: "0bc9a33b6ed226acd99e6cee842def02"
server_key_iv: "0c6ae6593577d135d501c42ed99a6002"
server_key_version: "encryptionkey@0"
I20260924 01:00:53.513221 27344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance:
uuid: "205155e209e3469eb9ff04215bd1c0c2"
format_stamp: "Formatted at 2026-09-24 01:00:53 on dist-test-slave-mmhs"
server_key: "0bc9a33b6ed226acd99e6cee842def02"
server_key_iv: "0c6ae6593577d135d501c42ed99a6002"
server_key_version: "encryptionkey@0"
I20260924 01:00:53.518563 27344 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260924 01:00:53.522130 27358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:53.523626 27344 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 01:00:53.523793 27344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-0/wal
uuid: "205155e209e3469eb9ff04215bd1c0c2"
format_stamp: "Formatted at 2026-09-24 01:00:53 on dist-test-slave-mmhs"
server_key: "0bc9a33b6ed226acd99e6cee842def02"
server_key_iv: "0c6ae6593577d135d501c42ed99a6002"
server_key_version: "encryptionkey@0"
I20260924 01:00:53.524024 27344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:00:53.539284 27344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:00:53.540071 27344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:00:53.540364 27344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:00:53.541363 27344 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 01:00:53.543043 27344 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 01:00:53.543129 27344 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:53.543273 27344 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 01:00:53.543357 27344 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:53.577826 27344 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.129:42141
I20260924 01:00:53.577886 27470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.129:42141 every 8 connection(s)
I20260924 01:00:53.579270 27344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
I20260924 01:00:53.579697 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 27344
I20260924 01:00:53.579876 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance
I20260924 01:00:53.580497 8450 external_mini_cluster.cc:1428] Setting key 21e3891144f80c86f3b446c4ae07c528
I20260924 01:00:53.583640 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.130:0
--local_ip_for_outbound_sockets=127.8.64.130
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:42981
--builtin_ntp_servers=127.8.64.148:45671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260924 01:00:53.595546 27473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:53.582190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:52371 (local address 127.8.64.190:42981)
0924 01:00:53.582669 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:53.582674 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:53.584167 (+ 1493us) server_negotiation.cc:407] Connection header received
0924 01:00:53.585940 (+ 1773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:53.585947 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:53.586039 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:53.586161 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:53.587322 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:53.588449 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:53.590000 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:53.590768 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:53.590907 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:53.594418 (+ 3511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:53.594427 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:53.594435 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:00:53.594462 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:53.594537 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:00:53.594556 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:53.594653 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:53.594929 (+ 276us) server_negotiation.cc:299] Negotiation successful
0924 01:00:53.595044 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":272,"threads_started":1}
I20260924 01:00:53.596675 27474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:53.582929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42981 (local address 127.8.64.129:52371)
0924 01:00:53.583993 (+ 1064us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:53.584044 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:53.585679 (+ 1635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:53.586483 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:53.586492 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:53.586580 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0924 01:00:53.587149 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:53.587165 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:53.588597 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:53.588601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:53.589852 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:53.589863 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:53.592940 (+ 3077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:53.592946 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:53.593192 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:53.594041 (+ 849us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 01:00:53.594044 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0924 01:00:53.594088 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 01:00:53.594174 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 01:00:53.594674 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 01:00:53.594685 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 01:00:53.594691 (+ 6us) client_negotiation.cc:769] Sending connection context
0924 01:00:53.594900 (+ 209us) client_negotiation.cc:240] Negotiation successful
0924 01:00:53.595391 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":369,"threads_started":1}
I20260924 01:00:53.598423 27471 heartbeater.cc:347] Connected to a master server at 127.8.64.190:42981
I20260924 01:00:53.598718 27471 heartbeater.cc:464] Registering TS with master...
I20260924 01:00:53.599470 27471 heartbeater.cc:511] Master 127.8.64.190:42981 requested a full tablet report, sending...
I20260924 01:00:53.601532 27285 ts_manager.cc:194] Registered new tserver with Master: 205155e209e3469eb9ff04215bd1c0c2 (127.8.64.129:42141)
I20260924 01:00:53.603044 27285 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.64.129:52371
W20260924 01:00:53.912453 27475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:00:53.912765 27475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:00:53.912916 27475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:00:53.920126 27475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:00:53.920277 27475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:00:53.920439 27475 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.130
I20260924 01:00:53.930258 27475 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:45671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:42981
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.130
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:00:53.931593 27475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:00:53.933395 27475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:00:53.946385 27480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:53.946483 27483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:53.946445 27481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:00:53.947479 27475 server_base.cc:1034] running on GCE node
I20260924 01:00:53.948143 27475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:00:53.949136 27475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:00:53.950399 27475 hybrid_clock.cc:648] HybridClock initialized: now 1790211653950338 us; error 49 us; skew 500 ppm
I20260924 01:00:53.950759 27475 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 01:00:53.953050 27475 webserver.cc:460] Webserver started at http://127.8.64.130:37011/ using document root <none> and password file <none>
I20260924 01:00:53.953729 27475 fs_manager.cc:359] Metadata directory not provided
I20260924 01:00:53.953858 27475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:00:53.954187 27475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:00:53.957078 27475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-1/data/instance:
uuid: "5ce9eb7ec3524118826ffe899029bd37"
format_stamp: "Formatted at 2026-09-24 01:00:53 on dist-test-slave-mmhs"
server_key: "65d3ff1770392780cf446430c39c5f14"
server_key_iv: "4e3fb1b66c2c35920e30edff9e8099b2"
server_key_version: "encryptionkey@0"
I20260924 01:00:53.957856 27475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance:
uuid: "5ce9eb7ec3524118826ffe899029bd37"
format_stamp: "Formatted at 2026-09-24 01:00:53 on dist-test-slave-mmhs"
server_key: "65d3ff1770392780cf446430c39c5f14"
server_key_iv: "4e3fb1b66c2c35920e30edff9e8099b2"
server_key_version: "encryptionkey@0"
I20260924 01:00:53.962826 27475 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260924 01:00:53.966248 27489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:53.967684 27475 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260924 01:00:53.967860 27475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-1/wal
uuid: "5ce9eb7ec3524118826ffe899029bd37"
format_stamp: "Formatted at 2026-09-24 01:00:53 on dist-test-slave-mmhs"
server_key: "65d3ff1770392780cf446430c39c5f14"
server_key_iv: "4e3fb1b66c2c35920e30edff9e8099b2"
server_key_version: "encryptionkey@0"
I20260924 01:00:53.968087 27475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:00:53.986975 27475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:00:53.987798 27475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:00:53.988112 27475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:00:53.989120 27475 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 01:00:53.990628 27475 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 01:00:53.990748 27475 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:53.990887 27475 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 01:00:53.990976 27475 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:54.025399 27475 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.130:33849
I20260924 01:00:54.025458 27601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.130:33849 every 8 connection(s)
I20260924 01:00:54.026788 27475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
I20260924 01:00:54.031651 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 27475
I20260924 01:00:54.031826 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance
I20260924 01:00:54.032266 8450 external_mini_cluster.cc:1428] Setting key 4ff9d53d5a130daae56e4e1ae9b6753e
I20260924 01:00:54.035386 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.131:0
--local_ip_for_outbound_sockets=127.8.64.131
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:42981
--builtin_ntp_servers=127.8.64.148:45671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260924 01:00:54.041565 27473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:54.029669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:60941 (local address 127.8.64.190:42981)
0924 01:00:54.029855 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:54.029860 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:54.031280 (+ 1420us) server_negotiation.cc:407] Connection header received
0924 01:00:54.033556 (+ 2276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:54.033561 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:54.033624 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:54.033702 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:54.035093 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:54.036238 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:54.037831 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:54.038524 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:54.038674 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:54.040557 (+ 1883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:54.040565 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:54.040573 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:00:54.040600 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:54.040643 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:00:54.040662 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:54.040760 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:54.041019 (+ 259us) server_negotiation.cc:299] Negotiation successful
0924 01:00:54.041142 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260924 01:00:54.042732 27604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:54.030133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42981 (local address 127.8.64.130:60941)
0924 01:00:54.031119 (+ 986us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:54.031166 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:54.033200 (+ 2034us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:54.033984 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:54.033996 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:54.034157 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0924 01:00:54.034912 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:54.034931 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:54.036392 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:54.036396 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:54.037675 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:54.037688 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:54.038890 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:54.038904 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:54.039304 (+ 400us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:54.040205 (+ 901us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 01:00:54.040208 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0924 01:00:54.040250 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 01:00:54.040335 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 01:00:54.040784 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 01:00:54.040796 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 01:00:54.040806 (+ 10us) client_negotiation.cc:769] Sending connection context
0924 01:00:54.040981 (+ 175us) client_negotiation.cc:240] Negotiation successful
0924 01:00:54.041397 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":345,"threads_started":1}
I20260924 01:00:54.044322 27602 heartbeater.cc:347] Connected to a master server at 127.8.64.190:42981
I20260924 01:00:54.044665 27602 heartbeater.cc:464] Registering TS with master...
I20260924 01:00:54.045446 27602 heartbeater.cc:511] Master 127.8.64.190:42981 requested a full tablet report, sending...
I20260924 01:00:54.047048 27285 ts_manager.cc:194] Registered new tserver with Master: 5ce9eb7ec3524118826ffe899029bd37 (127.8.64.130:33849)
I20260924 01:00:54.047907 27285 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.64.130:60941
W20260924 01:00:54.362958 27605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:00:54.363233 27605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:00:54.363384 27605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:00:54.371140 27605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:00:54.371248 27605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:00:54.371434 27605 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.131
I20260924 01:00:54.380513 27605 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:45671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:42981
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.131
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:00:54.381824 27605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:00:54.383617 27605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:00:54.397307 27610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:54.397228 27611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:54.397346 27613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:00:54.398025 27605 server_base.cc:1034] running on GCE node
I20260924 01:00:54.398777 27605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:00:54.399751 27605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:00:54.401015 27605 hybrid_clock.cc:648] HybridClock initialized: now 1790211654400946 us; error 65 us; skew 500 ppm
I20260924 01:00:54.401448 27605 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 01:00:54.403992 27605 webserver.cc:460] Webserver started at http://127.8.64.131:39615/ using document root <none> and password file <none>
I20260924 01:00:54.404698 27605 fs_manager.cc:359] Metadata directory not provided
I20260924 01:00:54.404834 27605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:00:54.405158 27605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:00:54.408028 27605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-2/data/instance:
uuid: "e4e512e614104bae9fc06a320c66efdf"
format_stamp: "Formatted at 2026-09-24 01:00:54 on dist-test-slave-mmhs"
server_key: "978ec12bbad26947aadf5e4600d40cff"
server_key_iv: "8a1f7d0f6fbbe2cf06216d6ffbc3c835"
server_key_version: "encryptionkey@0"
I20260924 01:00:54.408825 27605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance:
uuid: "e4e512e614104bae9fc06a320c66efdf"
format_stamp: "Formatted at 2026-09-24 01:00:54 on dist-test-slave-mmhs"
server_key: "978ec12bbad26947aadf5e4600d40cff"
server_key_iv: "8a1f7d0f6fbbe2cf06216d6ffbc3c835"
server_key_version: "encryptionkey@0"
I20260924 01:00:54.413913 27605 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260924 01:00:54.417433 27619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:54.419018 27605 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 01:00:54.419239 27605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-2/wal
uuid: "e4e512e614104bae9fc06a320c66efdf"
format_stamp: "Formatted at 2026-09-24 01:00:54 on dist-test-slave-mmhs"
server_key: "978ec12bbad26947aadf5e4600d40cff"
server_key_iv: "8a1f7d0f6fbbe2cf06216d6ffbc3c835"
server_key_version: "encryptionkey@0"
I20260924 01:00:54.419554 27605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:00:54.435807 27605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:00:54.436772 27605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:00:54.437139 27605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:00:54.438206 27605 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 01:00:54.439860 27605 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 01:00:54.440003 27605 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:54.440160 27605 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 01:00:54.440279 27605 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:54.480419 27605 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.131:42339
I20260924 01:00:54.480436 27731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.131:42339 every 8 connection(s)
I20260924 01:00:54.481977 27605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
I20260924 01:00:54.484916 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 27605
I20260924 01:00:54.485097 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance
I20260924 01:00:54.485597 8450 external_mini_cluster.cc:1428] Setting key bda4eb0190f8436d80f5746c2afe26d5
I20260924 01:00:54.495007 27473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:54.484843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:55807 (local address 127.8.64.190:42981)
0924 01:00:54.485095 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:54.485103 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:54.486729 (+ 1626us) server_negotiation.cc:407] Connection header received
0924 01:00:54.488969 (+ 2240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:54.488973 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:54.489049 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:54.489130 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:54.490133 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:54.490967 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:54.492433 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:54.492849 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:54.492943 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:54.494239 (+ 1296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:54.494244 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:54.494247 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:00:54.494264 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:54.494343 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:00:54.494359 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:54.494439 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:54.494698 (+ 259us) server_negotiation.cc:299] Negotiation successful
0924 01:00:54.494769 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260924 01:00:54.496037 27734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:54.485403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:42981 (local address 127.8.64.131:55807)
0924 01:00:54.486541 (+ 1138us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:54.486599 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:54.488730 (+ 2131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:54.489328 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:54.489336 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:54.489421 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0924 01:00:54.489966 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:54.489998 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:54.491098 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:54.491102 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:54.492305 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:54.492316 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:54.493038 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:54.493043 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:54.493234 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:54.493942 (+ 708us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 01:00:54.493945 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0924 01:00:54.493992 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 01:00:54.494065 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 01:00:54.494494 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 01:00:54.494502 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 01:00:54.494508 (+ 6us) client_negotiation.cc:769] Sending connection context
0924 01:00:54.494657 (+ 149us) client_negotiation.cc:240] Negotiation successful
0924 01:00:54.494940 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":343,"threads_started":1}
I20260924 01:00:54.497399 27732 heartbeater.cc:347] Connected to a master server at 127.8.64.190:42981
I20260924 01:00:54.497701 27732 heartbeater.cc:464] Registering TS with master...
I20260924 01:00:54.498498 27732 heartbeater.cc:511] Master 127.8.64.190:42981 requested a full tablet report, sending...
I20260924 01:00:54.500097 27285 ts_manager.cc:194] Registered new tserver with Master: e4e512e614104bae9fc06a320c66efdf (127.8.64.131:42339)
I20260924 01:00:54.501166 8450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 01:00:54.501315 27285 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.64.131:55807
I20260924 01:00:54.511776 27473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:54.505962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38932 (local address 127.8.64.190:42981)
0924 01:00:54.506156 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:54.506160 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:54.506597 (+ 437us) server_negotiation.cc:407] Connection header received
0924 01:00:54.506698 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:54.506702 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:54.506760 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:54.506844 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:54.507287 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:54.507994 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:54.509293 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:54.509778 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:54.509897 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:54.511082 (+ 1185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:54.511086 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:54.511090 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:00:54.511108 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:54.511139 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:00:54.511151 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:54.511270 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:54.511483 (+ 213us) server_negotiation.cc:299] Negotiation successful
0924 01:00:54.511530 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260924 01:00:54.515727 27285 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38932:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 01:00:54.522578 27285 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 01:00:54.555123 27745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:54.535221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38864 (local address 127.8.64.130:33849)
0924 01:00:54.535770 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:54.535779 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:54.536000 (+ 221us) server_negotiation.cc:407] Connection header received
0924 01:00:54.542991 (+ 6991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:54.543010 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:54.544347 (+ 1337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:54.544704 (+ 357us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 01:00:54.545790 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:54.548083 (+ 2293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:54.551919 (+ 3836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:54.553569 (+ 1650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:54.553709 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:54.553971 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:54.554726 (+ 755us) server_negotiation.cc:299] Negotiation successful
0924 01:00:54.554862 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":240,"threads_started":1}
I20260924 01:00:54.555814 27744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:54.535082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.130:33849 (local address 127.0.0.1:38864)
0924 01:00:54.535869 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:54.535905 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:54.536133 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:54.544757 (+ 8624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:54.544771 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:54.544830 (+ 59us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 01:00:54.545422 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:54.545441 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:54.548262 (+ 2821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:54.548271 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:54.551621 (+ 3350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:54.551647 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:54.553800 (+ 2153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:54.553807 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:54.554046 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:54.554103 (+ 57us) client_negotiation.cc:769] Sending connection context
0924 01:00:54.554244 (+ 141us) client_negotiation.cc:240] Negotiation successful
0924 01:00:54.554513 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":371,"threads_started":1}
I20260924 01:00:54.560256 27747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:54.536430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:42141 (local address 127.0.0.1:43776)
0924 01:00:54.548416 (+ 11986us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:54.548447 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:54.548629 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:54.551882 (+ 3253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:54.551888 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:54.551930 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 01:00:54.552379 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:54.552394 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:54.554559 (+ 2165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:54.554564 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:54.557701 (+ 3137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:54.557713 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:54.559492 (+ 1779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:54.559498 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:54.559683 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:54.559716 (+ 33us) client_negotiation.cc:769] Sending connection context
0924 01:00:54.559805 (+ 89us) client_negotiation.cc:240] Negotiation successful
0924 01:00:54.559954 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11730,"thread_start_us":16181,"threads_started":1}
I20260924 01:00:54.560746 27748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:54.536647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43776 (local address 127.8.64.129:42141)
0924 01:00:54.549311 (+ 12664us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:54.549327 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:54.549373 (+ 46us) server_negotiation.cc:407] Connection header received
0924 01:00:54.549515 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:54.549529 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:54.549916 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:54.550187 (+ 271us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 01:00:54.552631 (+ 2444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:54.554391 (+ 1760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:54.557917 (+ 3526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:54.558964 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:54.559102 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:54.559361 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:54.560217 (+ 856us) server_negotiation.cc:299] Negotiation successful
0924 01:00:54.560432 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12414,"thread_start_us":16545,"threads_started":1}
I20260924 01:00:54.560463 27537 tablet_service.cc:1467] Processing CreateTablet for tablet 812e91b8f15c4ef3a6e85e5792d5ab38 (DEFAULT_TABLE table=test-table [id=0e9723b9e36f4aaaa12ba4e8a97c00b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 01:00:54.562083 27537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 812e91b8f15c4ef3a6e85e5792d5ab38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:00:54.562148 27746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:54.536548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.131:42339 (local address 127.0.0.1:59092)
0924 01:00:54.552916 (+ 16368us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:54.552938 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:54.553080 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:54.554065 (+ 985us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:54.554094 (+ 29us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:54.554123 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 01:00:54.554577 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:54.554592 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:54.557492 (+ 2900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:54.557499 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:54.559585 (+ 2086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:54.559600 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:54.561385 (+ 1785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:54.561392 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:54.561558 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:54.561602 (+ 44us) client_negotiation.cc:769] Sending connection context
0924 01:00:54.561690 (+ 88us) client_negotiation.cc:240] Negotiation successful
0924 01:00:54.561825 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16226,"thread_start_us":13921,"threads_started":1}
I20260924 01:00:54.562346 27749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:54.536625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59092 (local address 127.8.64.131:42339)
0924 01:00:54.552452 (+ 15827us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:54.552464 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:54.553070 (+ 606us) server_negotiation.cc:407] Connection header received
0924 01:00:54.553225 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:54.553233 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:54.553490 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:54.553715 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 01:00:54.555671 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:54.557340 (+ 1669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:54.559820 (+ 2480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:54.561195 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:54.561340 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:54.561605 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:54.561747 (+ 142us) server_negotiation.cc:299] Negotiation successful
0924 01:00:54.561912 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15344,"thread_start_us":15553,"threads_started":1}
I20260924 01:00:54.564585 27406 tablet_service.cc:1467] Processing CreateTablet for tablet 812e91b8f15c4ef3a6e85e5792d5ab38 (DEFAULT_TABLE table=test-table [id=0e9723b9e36f4aaaa12ba4e8a97c00b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 01:00:54.565825 27406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 812e91b8f15c4ef3a6e85e5792d5ab38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:00:54.565876 27667 tablet_service.cc:1467] Processing CreateTablet for tablet 812e91b8f15c4ef3a6e85e5792d5ab38 (DEFAULT_TABLE table=test-table [id=0e9723b9e36f4aaaa12ba4e8a97c00b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 01:00:54.567293 27667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 812e91b8f15c4ef3a6e85e5792d5ab38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:00:54.577391 27750 tablet_bootstrap.cc:492] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 5ce9eb7ec3524118826ffe899029bd37: Bootstrap starting.
I20260924 01:00:54.584162 27750 tablet_bootstrap.cc:654] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 5ce9eb7ec3524118826ffe899029bd37: Neither blocks nor log segments found. Creating new log.
I20260924 01:00:54.585934 27750 log.cc:824] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 5ce9eb7ec3524118826ffe899029bd37: Log is configured to *not* fsync() on all Append() calls
I20260924 01:00:54.595013 27751 tablet_bootstrap.cc:492] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 205155e209e3469eb9ff04215bd1c0c2: Bootstrap starting.
I20260924 01:00:54.595917 27752 tablet_bootstrap.cc:492] T 812e91b8f15c4ef3a6e85e5792d5ab38 P e4e512e614104bae9fc06a320c66efdf: Bootstrap starting.
I20260924 01:00:54.598996 27750 tablet_bootstrap.cc:492] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 5ce9eb7ec3524118826ffe899029bd37: No bootstrap required, opened a new log
I20260924 01:00:54.599298 27750 ts_tablet_manager.cc:1397] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 5ce9eb7ec3524118826ffe899029bd37: Time spent bootstrapping tablet: real 0.022s user 0.009s sys 0.008s
I20260924 01:00:54.600327 27752 tablet_bootstrap.cc:654] T 812e91b8f15c4ef3a6e85e5792d5ab38 P e4e512e614104bae9fc06a320c66efdf: Neither blocks nor log segments found. Creating new log.
I20260924 01:00:54.601142 27751 tablet_bootstrap.cc:654] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 205155e209e3469eb9ff04215bd1c0c2: Neither blocks nor log segments found. Creating new log.
I20260924 01:00:54.601580 27752 log.cc:824] T 812e91b8f15c4ef3a6e85e5792d5ab38 P e4e512e614104bae9fc06a320c66efdf: Log is configured to *not* fsync() on all Append() calls
I20260924 01:00:54.602130 27751 log.cc:824] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 205155e209e3469eb9ff04215bd1c0c2: Log is configured to *not* fsync() on all Append() calls
I20260924 01:00:54.605211 27752 tablet_bootstrap.cc:492] T 812e91b8f15c4ef3a6e85e5792d5ab38 P e4e512e614104bae9fc06a320c66efdf: No bootstrap required, opened a new log
I20260924 01:00:54.605266 27751 tablet_bootstrap.cc:492] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 205155e209e3469eb9ff04215bd1c0c2: No bootstrap required, opened a new log
I20260924 01:00:54.605499 27752 ts_tablet_manager.cc:1397] T 812e91b8f15c4ef3a6e85e5792d5ab38 P e4e512e614104bae9fc06a320c66efdf: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260924 01:00:54.605571 27751 ts_tablet_manager.cc:1397] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 205155e209e3469eb9ff04215bd1c0c2: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.003s
I20260924 01:00:54.605970 27471 heartbeater.cc:503] Master 127.8.64.190:42981 was elected leader, sending a full tablet report...
I20260924 01:00:54.610136 27750 raft_consensus.cc:348] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 5ce9eb7ec3524118826ffe899029bd37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce9eb7ec3524118826ffe899029bd37" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 33849 } } peers { permanent_uuid: "e4e512e614104bae9fc06a320c66efdf" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 42339 } } peers { permanent_uuid: "205155e209e3469eb9ff04215bd1c0c2" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 42141 } }
I20260924 01:00:54.610728 27750 raft_consensus.cc:374] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 5ce9eb7ec3524118826ffe899029bd37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 01:00:54.610838 27750 raft_consensus.cc:729] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 5ce9eb7ec3524118826ffe899029bd37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce9eb7ec3524118826ffe899029bd37, State: Initialized, Role: FOLLOWER
I20260924 01:00:54.611464 27750 consensus_queue.cc:260] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 5ce9eb7ec3524118826ffe899029bd37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce9eb7ec3524118826ffe899029bd37" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 33849 } } peers { permanent_uuid: "e4e512e614104bae9fc06a320c66efdf" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 42339 } } peers { permanent_uuid: "205155e209e3469eb9ff04215bd1c0c2" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 42141 } }
I20260924 01:00:54.612584 27602 heartbeater.cc:503] Master 127.8.64.190:42981 was elected leader, sending a full tablet report...
I20260924 01:00:54.613241 27750 ts_tablet_manager.cc:1428] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 5ce9eb7ec3524118826ffe899029bd37: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260924 01:00:54.615054 27752 raft_consensus.cc:348] T 812e91b8f15c4ef3a6e85e5792d5ab38 P e4e512e614104bae9fc06a320c66efdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce9eb7ec3524118826ffe899029bd37" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 33849 } } peers { permanent_uuid: "e4e512e614104bae9fc06a320c66efdf" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 42339 } } peers { permanent_uuid: "205155e209e3469eb9ff04215bd1c0c2" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 42141 } }
I20260924 01:00:54.615545 27752 raft_consensus.cc:374] T 812e91b8f15c4ef3a6e85e5792d5ab38 P e4e512e614104bae9fc06a320c66efdf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 01:00:54.615672 27752 raft_consensus.cc:729] T 812e91b8f15c4ef3a6e85e5792d5ab38 P e4e512e614104bae9fc06a320c66efdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4e512e614104bae9fc06a320c66efdf, State: Initialized, Role: FOLLOWER
I20260924 01:00:54.615803 27751 raft_consensus.cc:348] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 205155e209e3469eb9ff04215bd1c0c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce9eb7ec3524118826ffe899029bd37" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 33849 } } peers { permanent_uuid: "e4e512e614104bae9fc06a320c66efdf" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 42339 } } peers { permanent_uuid: "205155e209e3469eb9ff04215bd1c0c2" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 42141 } }
I20260924 01:00:54.616278 27751 raft_consensus.cc:374] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 205155e209e3469eb9ff04215bd1c0c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 01:00:54.616415 27751 raft_consensus.cc:729] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 205155e209e3469eb9ff04215bd1c0c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 205155e209e3469eb9ff04215bd1c0c2, State: Initialized, Role: FOLLOWER
I20260924 01:00:54.616326 27752 consensus_queue.cc:260] T 812e91b8f15c4ef3a6e85e5792d5ab38 P e4e512e614104bae9fc06a320c66efdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce9eb7ec3524118826ffe899029bd37" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 33849 } } peers { permanent_uuid: "e4e512e614104bae9fc06a320c66efdf" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 42339 } } peers { permanent_uuid: "205155e209e3469eb9ff04215bd1c0c2" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 42141 } }
I20260924 01:00:54.617046 27751 consensus_queue.cc:260] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 205155e209e3469eb9ff04215bd1c0c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce9eb7ec3524118826ffe899029bd37" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 33849 } } peers { permanent_uuid: "e4e512e614104bae9fc06a320c66efdf" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 42339 } } peers { permanent_uuid: "205155e209e3469eb9ff04215bd1c0c2" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 42141 } }
I20260924 01:00:54.617751 27732 heartbeater.cc:503] Master 127.8.64.190:42981 was elected leader, sending a full tablet report...
I20260924 01:00:54.618350 27752 ts_tablet_manager.cc:1428] T 812e91b8f15c4ef3a6e85e5792d5ab38 P e4e512e614104bae9fc06a320c66efdf: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260924 01:00:54.623334 27751 ts_tablet_manager.cc:1428] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 205155e209e3469eb9ff04215bd1c0c2: Time spent starting tablet: real 0.018s user 0.012s sys 0.005s
I20260924 01:00:54.649449 27758 raft_consensus.cc:482] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 205155e209e3469eb9ff04215bd1c0c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 01:00:54.649741 27758 raft_consensus.cc:504] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 205155e209e3469eb9ff04215bd1c0c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce9eb7ec3524118826ffe899029bd37" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 33849 } } peers { permanent_uuid: "e4e512e614104bae9fc06a320c66efdf" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 42339 } } peers { permanent_uuid: "205155e209e3469eb9ff04215bd1c0c2" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 42141 } }
I20260924 01:00:54.651737 27758 leader_election.cc:288] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 205155e209e3469eb9ff04215bd1c0c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce9eb7ec3524118826ffe899029bd37 (127.8.64.130:33849), e4e512e614104bae9fc06a320c66efdf (127.8.64.131:42339)
I20260924 01:00:54.659077 27745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:54.652113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:44983 (local address 127.8.64.130:33849)
0924 01:00:54.652345 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:54.652353 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:54.652927 (+ 574us) server_negotiation.cc:407] Connection header received
0924 01:00:54.653217 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:54.653223 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:54.653318 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:54.653412 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 01:00:54.654139 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:54.655096 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:54.657162 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:54.658038 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:54.658159 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:54.658208 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:54.658665 (+ 457us) server_negotiation.cc:299] Negotiation successful
0924 01:00:54.658805 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260924 01:00:54.659152 27759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:54.652011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.130:33849 (local address 127.8.64.129:44983)
0924 01:00:54.652820 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:54.652839 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:54.653058 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:54.653512 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:54.653516 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:54.653538 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 01:00:54.653991 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:54.654005 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:54.655228 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:54.655234 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:54.656977 (+ 1743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:54.656989 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:54.658227 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:54.658233 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:54.658429 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:54.658527 (+ 98us) client_negotiation.cc:769] Sending connection context
0924 01:00:54.658606 (+ 79us) client_negotiation.cc:240] Negotiation successful
0924 01:00:54.658693 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"mutex_wait_us":156,"spinlock_wait_cycles":37504,"thread_start_us":449,"threads_started":1}
I20260924 01:00:54.659170 27749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:54.653185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:39401 (local address 127.8.64.131:42339)
0924 01:00:54.653405 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:54.653411 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:54.653725 (+ 314us) server_negotiation.cc:407] Connection header received
0924 01:00:54.653881 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:54.653886 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:54.653976 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:54.654078 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 01:00:54.654713 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:54.655553 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:54.657403 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:54.658213 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:54.658393 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:54.658457 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:54.658796 (+ 339us) server_negotiation.cc:299] Negotiation successful
0924 01:00:54.658919 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260924 01:00:54.659466 27760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:54.652881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.131:42339 (local address 127.8.64.129:39401)
0924 01:00:54.653617 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:54.653632 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:54.653750 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:54.654122 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:54.654126 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:54.654149 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 01:00:54.654565 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:54.654578 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:54.655695 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:54.655699 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:54.657226 (+ 1527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:54.657240 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:54.658450 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:54.658454 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:54.658633 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:54.658663 (+ 30us) client_negotiation.cc:769] Sending connection context
0924 01:00:54.658750 (+ 87us) client_negotiation.cc:240] Negotiation successful
0924 01:00:54.658857 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"mutex_wait_us":49,"spinlock_wait_cycles":896,"thread_start_us":410,"threads_started":1}
I20260924 01:00:54.660130 27557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "812e91b8f15c4ef3a6e85e5792d5ab38" candidate_uuid: "205155e209e3469eb9ff04215bd1c0c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce9eb7ec3524118826ffe899029bd37" is_pre_election: true
I20260924 01:00:54.660434 27687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "812e91b8f15c4ef3a6e85e5792d5ab38" candidate_uuid: "205155e209e3469eb9ff04215bd1c0c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4e512e614104bae9fc06a320c66efdf" is_pre_election: true
I20260924 01:00:54.660624 27557 raft_consensus.cc:2454] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 5ce9eb7ec3524118826ffe899029bd37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 205155e209e3469eb9ff04215bd1c0c2 in term 0.
I20260924 01:00:54.660919 27687 raft_consensus.cc:2454] T 812e91b8f15c4ef3a6e85e5792d5ab38 P e4e512e614104bae9fc06a320c66efdf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 205155e209e3469eb9ff04215bd1c0c2 in term 0.
I20260924 01:00:54.661293 27362 leader_election.cc:302] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 205155e209e3469eb9ff04215bd1c0c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 205155e209e3469eb9ff04215bd1c0c2, 5ce9eb7ec3524118826ffe899029bd37; no voters:
I20260924 01:00:54.661659 27758 raft_consensus.cc:2781] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 205155e209e3469eb9ff04215bd1c0c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 01:00:54.661801 27758 raft_consensus.cc:482] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 205155e209e3469eb9ff04215bd1c0c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 01:00:54.661943 27758 raft_consensus.cc:3037] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 205155e209e3469eb9ff04215bd1c0c2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:00:54.664124 27758 raft_consensus.cc:504] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 205155e209e3469eb9ff04215bd1c0c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce9eb7ec3524118826ffe899029bd37" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 33849 } } peers { permanent_uuid: "e4e512e614104bae9fc06a320c66efdf" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 42339 } } peers { permanent_uuid: "205155e209e3469eb9ff04215bd1c0c2" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 42141 } }
I20260924 01:00:54.664665 27758 leader_election.cc:288] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 205155e209e3469eb9ff04215bd1c0c2 [CANDIDATE]: Term 1 election: Requested vote from peers 5ce9eb7ec3524118826ffe899029bd37 (127.8.64.130:33849), e4e512e614104bae9fc06a320c66efdf (127.8.64.131:42339)
I20260924 01:00:54.665253 27557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "812e91b8f15c4ef3a6e85e5792d5ab38" candidate_uuid: "205155e209e3469eb9ff04215bd1c0c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce9eb7ec3524118826ffe899029bd37"
I20260924 01:00:54.665254 27687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "812e91b8f15c4ef3a6e85e5792d5ab38" candidate_uuid: "205155e209e3469eb9ff04215bd1c0c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4e512e614104bae9fc06a320c66efdf"
I20260924 01:00:54.665468 27687 raft_consensus.cc:3037] T 812e91b8f15c4ef3a6e85e5792d5ab38 P e4e512e614104bae9fc06a320c66efdf [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:00:54.665468 27557 raft_consensus.cc:3037] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 5ce9eb7ec3524118826ffe899029bd37 [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:00:54.668710 27557 raft_consensus.cc:2454] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 5ce9eb7ec3524118826ffe899029bd37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 205155e209e3469eb9ff04215bd1c0c2 in term 1.
I20260924 01:00:54.668710 27687 raft_consensus.cc:2454] T 812e91b8f15c4ef3a6e85e5792d5ab38 P e4e512e614104bae9fc06a320c66efdf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 205155e209e3469eb9ff04215bd1c0c2 in term 1.
I20260924 01:00:54.669086 27362 leader_election.cc:302] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 205155e209e3469eb9ff04215bd1c0c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 205155e209e3469eb9ff04215bd1c0c2, 5ce9eb7ec3524118826ffe899029bd37; no voters:
I20260924 01:00:54.669343 27758 raft_consensus.cc:2781] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 205155e209e3469eb9ff04215bd1c0c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 01:00:54.669796 27758 raft_consensus.cc:686] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 205155e209e3469eb9ff04215bd1c0c2 [term 1 LEADER]: Becoming Leader. State: Replica: 205155e209e3469eb9ff04215bd1c0c2, State: Running, Role: LEADER
I20260924 01:00:54.670378 27758 consensus_queue.cc:237] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 205155e209e3469eb9ff04215bd1c0c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce9eb7ec3524118826ffe899029bd37" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 33849 } } peers { permanent_uuid: "e4e512e614104bae9fc06a320c66efdf" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 42339 } } peers { permanent_uuid: "205155e209e3469eb9ff04215bd1c0c2" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 42141 } }
I20260924 01:00:54.675197 27284 catalog_manager.cc:5495] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 205155e209e3469eb9ff04215bd1c0c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 205155e209e3469eb9ff04215bd1c0c2 (127.8.64.129). New cstate: current_term: 1 leader_uuid: "205155e209e3469eb9ff04215bd1c0c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce9eb7ec3524118826ffe899029bd37" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 33849 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4e512e614104bae9fc06a320c66efdf" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 42339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "205155e209e3469eb9ff04215bd1c0c2" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 42141 } health_report { overall_health: HEALTHY } } }
I20260924 01:00:54.719833 27748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:54.714524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43790 (local address 127.8.64.129:42141)
0924 01:00:54.714713 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:54.714717 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:54.714735 (+ 18us) server_negotiation.cc:407] Connection header received
0924 01:00:54.714917 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:54.714921 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:54.715009 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:54.715138 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 01:00:54.715485 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:54.716193 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:54.717694 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:54.718118 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:54.718216 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:54.718807 (+ 591us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 01:00:54.719314 (+ 507us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 01:00:54.719441 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:54.719585 (+ 144us) server_negotiation.cc:299] Negotiation successful
0924 01:00:54.719623 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260924 01:00:54.728048 27557 raft_consensus.cc:1261] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 5ce9eb7ec3524118826ffe899029bd37 [term 1 FOLLOWER]: Refusing update from remote peer 205155e209e3469eb9ff04215bd1c0c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 01:00:54.728048 27687 raft_consensus.cc:1261] T 812e91b8f15c4ef3a6e85e5792d5ab38 P e4e512e614104bae9fc06a320c66efdf [term 1 FOLLOWER]: Refusing update from remote peer 205155e209e3469eb9ff04215bd1c0c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 01:00:54.729066 27761 consensus_queue.cc:1035] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 205155e209e3469eb9ff04215bd1c0c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ce9eb7ec3524118826ffe899029bd37" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 33849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 01:00:54.729450 27758 consensus_queue.cc:1035] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 205155e209e3469eb9ff04215bd1c0c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4e512e614104bae9fc06a320c66efdf" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 42339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260924 01:00:54.734023 27733 tablet.cc:2369] T 812e91b8f15c4ef3a6e85e5792d5ab38 P e4e512e614104bae9fc06a320c66efdf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 01:00:54.741500 27765 mvcc.cc:205] Tried to move back new op lower bound from 7332706937755820032 to 7332706937533919232. Current Snapshot: MvccSnapshot[applied={T|T < 7332706937755820032}]
I20260924 01:00:54.761339 27284 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 01:00:54.761637 27284 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 01:00:54.765334 27284 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2ff8d2ed07864ed580faf26b100e581a: Sending DeleteTablet for 3 replicas of tablet 812e91b8f15c4ef3a6e85e5792d5ab38
I20260924 01:00:54.766587 27406 tablet_service.cc:1514] Processing DeleteTablet for tablet 812e91b8f15c4ef3a6e85e5792d5ab38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 01:00:54 UTC) from {username='slave'} at 127.0.0.1:43776
I20260924 01:00:54.766873 27537 tablet_service.cc:1514] Processing DeleteTablet for tablet 812e91b8f15c4ef3a6e85e5792d5ab38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 01:00:54 UTC) from {username='slave'} at 127.0.0.1:38864
I20260924 01:00:54.766978 27667 tablet_service.cc:1514] Processing DeleteTablet for tablet 812e91b8f15c4ef3a6e85e5792d5ab38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 01:00:54 UTC) from {username='slave'} at 127.0.0.1:59092
I20260924 01:00:54.767733 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 27344
I20260924 01:00:54.776950 27774 tablet_replica.cc:330] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 205155e209e3469eb9ff04215bd1c0c2: stopping tablet replica
I20260924 01:00:54.777747 27775 tablet_replica.cc:330] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 5ce9eb7ec3524118826ffe899029bd37: stopping tablet replica
I20260924 01:00:54.778012 27774 raft_consensus.cc:2229] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 205155e209e3469eb9ff04215bd1c0c2 [term 1 LEADER]: Raft consensus shutting down.
I20260924 01:00:54.778461 27775 raft_consensus.cc:2229] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 5ce9eb7ec3524118826ffe899029bd37 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 01:00:54.779163 27775 raft_consensus.cc:2258] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 5ce9eb7ec3524118826ffe899029bd37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 01:00:54.779424 27774 raft_consensus.cc:2258] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 205155e209e3469eb9ff04215bd1c0c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 01:00:54.781589 27775 ts_tablet_manager.cc:1905] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 5ce9eb7ec3524118826ffe899029bd37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 01:00:54.782209 27774 ts_tablet_manager.cc:1905] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 205155e209e3469eb9ff04215bd1c0c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 01:00:54.786222 27775 ts_tablet_manager.cc:1918] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 5ce9eb7ec3524118826ffe899029bd37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 01:00:54.786408 27775 log.cc:1196] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 5ce9eb7ec3524118826ffe899029bd37: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-1/wal/wals/812e91b8f15c4ef3a6e85e5792d5ab38
I20260924 01:00:54.787020 27775 ts_tablet_manager.cc:1939] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 5ce9eb7ec3524118826ffe899029bd37: Deleting consensus metadata
I20260924 01:00:54.788898 27272 catalog_manager.cc:4841] TS 5ce9eb7ec3524118826ffe899029bd37 (127.8.64.130:33849): tablet 812e91b8f15c4ef3a6e85e5792d5ab38 (table test-table [id=0e9723b9e36f4aaaa12ba4e8a97c00b8]) successfully deleted
I20260924 01:00:54.789453 27774 ts_tablet_manager.cc:1918] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 205155e209e3469eb9ff04215bd1c0c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 01:00:54.789783 27774 log.cc:1196] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 205155e209e3469eb9ff04215bd1c0c2: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-0/wal/wals/812e91b8f15c4ef3a6e85e5792d5ab38
I20260924 01:00:54.789785 27776 tablet_replica.cc:330] T 812e91b8f15c4ef3a6e85e5792d5ab38 P e4e512e614104bae9fc06a320c66efdf: stopping tablet replica
I20260924 01:00:54.790233 27776 raft_consensus.cc:2229] T 812e91b8f15c4ef3a6e85e5792d5ab38 P e4e512e614104bae9fc06a320c66efdf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 01:00:54.790465 27774 ts_tablet_manager.cc:1939] T 812e91b8f15c4ef3a6e85e5792d5ab38 P 205155e209e3469eb9ff04215bd1c0c2: Deleting consensus metadata
I20260924 01:00:54.790805 27776 raft_consensus.cc:2258] T 812e91b8f15c4ef3a6e85e5792d5ab38 P e4e512e614104bae9fc06a320c66efdf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 01:00:54.792810 27269 catalog_manager.cc:4841] TS 205155e209e3469eb9ff04215bd1c0c2 (127.8.64.129:42141): tablet 812e91b8f15c4ef3a6e85e5792d5ab38 (table test-table [id=0e9723b9e36f4aaaa12ba4e8a97c00b8]) successfully deleted
I20260924 01:00:54.793401 27776 ts_tablet_manager.cc:1905] T 812e91b8f15c4ef3a6e85e5792d5ab38 P e4e512e614104bae9fc06a320c66efdf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 01:00:54.794368 27748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:54.769849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43806 (local address 127.8.64.129:42141)
0924 01:00:54.770068 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:54.770077 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:54.780256 (+ 10179us) server_negotiation.cc:407] Connection header received
0924 01:00:54.780447 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:54.780452 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:54.780582 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:54.780690 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:54.787109 (+ 6419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:54.788352 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:54.790231 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:54.790922 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:54.791054 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:54.792587 (+ 1533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:54.792597 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:54.792614 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:00:54.793424 (+ 810us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:54.793500 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:00:54.793526 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:54.793631 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:54.793811 (+ 180us) server_negotiation.cc:299] Negotiation successful
0924 01:00:54.793915 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260924 01:00:54.795094 27466 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 01:00:54.800808 27776 ts_tablet_manager.cc:1918] T 812e91b8f15c4ef3a6e85e5792d5ab38 P e4e512e614104bae9fc06a320c66efdf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 01:00:54.800968 27776 log.cc:1196] T 812e91b8f15c4ef3a6e85e5792d5ab38 P e4e512e614104bae9fc06a320c66efdf: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790211343099518-8450-0/minicluster-data/ts-2/wal/wals/812e91b8f15c4ef3a6e85e5792d5ab38
I20260924 01:00:54.801468 27776 ts_tablet_manager.cc:1939] T 812e91b8f15c4ef3a6e85e5792d5ab38 P e4e512e614104bae9fc06a320c66efdf: Deleting consensus metadata
I20260924 01:00:54.803279 27272 catalog_manager.cc:4841] TS e4e512e614104bae9fc06a320c66efdf (127.8.64.131:42339): tablet 812e91b8f15c4ef3a6e85e5792d5ab38 (table test-table [id=0e9723b9e36f4aaaa12ba4e8a97c00b8]) successfully deleted
I20260924 01:00:54.891791 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 27344
I20260924 01:00:54.911198 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 27475
I20260924 01:00:54.918632 27745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:54.912017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38866 (local address 127.8.64.130:33849)
0924 01:00:54.912234 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:54.912240 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:54.912261 (+ 21us) server_negotiation.cc:407] Connection header received
0924 01:00:54.912524 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:54.912528 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:54.912625 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:54.912736 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:54.913152 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:54.913922 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:54.915278 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:54.915687 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:54.915818 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:54.916974 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:54.916983 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:54.916997 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:00:54.917617 (+ 620us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:54.917666 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:00:54.917686 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:54.917819 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:54.917952 (+ 133us) server_negotiation.cc:299] Negotiation successful
0924 01:00:54.918060 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260924 01:00:54.919323 27597 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 01:00:55.009691 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 27475
I20260924 01:00:55.026981 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 27605
I20260924 01:00:55.033931 27749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:55.027766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59108 (local address 127.8.64.131:42339)
0924 01:00:55.027962 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:55.027970 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:55.027996 (+ 26us) server_negotiation.cc:407] Connection header received
0924 01:00:55.028211 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:55.028217 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:55.028327 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:55.028468 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:55.028846 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:55.029592 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:55.031028 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:55.031439 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:55.031579 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:55.032602 (+ 1023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:55.032611 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:55.032627 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:00:55.033181 (+ 554us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:55.033228 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:00:55.033251 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:55.033379 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:55.033518 (+ 139us) server_negotiation.cc:299] Negotiation successful
0924 01:00:55.033626 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260924 01:00:55.034639 27727 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 01:00:55.122411 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 27605
I20260924 01:00:55.139431 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 27254
I20260924 01:00:55.140477 27315 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 01:00:55.253477 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 27254
2026-09-24T01:00:55Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2753 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 01:00:55 dist-test-slave-mmhs krb5kdc[27784](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 01:00:55 dist-test-slave-mmhs krb5kdc[27784](info): set up 2 sockets
Sep 24 01:00:55 dist-test-slave-mmhs krb5kdc[27784](info): commencing operation
krb5kdc: starting...
W20260924 01:00:56.396132 8450 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.006s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Sep 24 01:00:56 dist-test-slave-mmhs krb5kdc[27784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T01:00:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T01:00:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.64.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
I20260924 01:00:56.682952 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:38985
--webserver_interface=127.8.64.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:40193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:38985
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 01:00:57.008996 27797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:00:57.009268 27797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:00:57.009362 27797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:00:57.016690 27797 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 01:00:57.016805 27797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:00:57.016852 27797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:00:57.016911 27797 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 01:00:57.016954 27797 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 01:00:57.026794 27797 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:40193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:38985
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:38985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:00:57.028074 27797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:00:57.029326 27797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:00:57.041241 27805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:57.041296 27803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:57.041322 27802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:00:57.042091 27797 server_base.cc:1034] running on GCE node
I20260924 01:00:57.042975 27797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:00:57.044366 27797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:00:57.045585 27797 hybrid_clock.cc:648] HybridClock initialized: now 1790211657045522 us; error 43 us; skew 500 ppm
I20260924 01:00:57.045944 27797 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 01:00:57 dist-test-slave-mmhs krb5kdc[27784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211657, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 01:00:57.049006 27797 init.cc:377] Logged in from keytab as kudu/127.8.64.190@KRBTEST.COM (short username kudu)
I20260924 01:00:57.050642 27797 webserver.cc:460] Webserver started at http://127.8.64.190:45087/ using document root <none> and password file <none>
I20260924 01:00:57.051326 27797 fs_manager.cc:359] Metadata directory not provided
I20260924 01:00:57.051470 27797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:00:57.051781 27797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:00:57.054850 27797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "0b2d4d8c2f9a400998ef07b4860b3bb8"
format_stamp: "Formatted at 2026-09-24 01:00:57 on dist-test-slave-mmhs"
server_key: "7ccbbce9f24800727a035cd8c5bfe00e"
server_key_iv: "11684a0f8223f8ae38cbb2cdfe9e9f4e"
server_key_version: "encryptionkey@0"
I20260924 01:00:57.055609 27797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "0b2d4d8c2f9a400998ef07b4860b3bb8"
format_stamp: "Formatted at 2026-09-24 01:00:57 on dist-test-slave-mmhs"
server_key: "7ccbbce9f24800727a035cd8c5bfe00e"
server_key_iv: "11684a0f8223f8ae38cbb2cdfe9e9f4e"
server_key_version: "encryptionkey@0"
I20260924 01:00:57.060796 27797 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260924 01:00:57.064252 27812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:57.065585 27797 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260924 01:00:57.065789 27797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "0b2d4d8c2f9a400998ef07b4860b3bb8"
format_stamp: "Formatted at 2026-09-24 01:00:57 on dist-test-slave-mmhs"
server_key: "7ccbbce9f24800727a035cd8c5bfe00e"
server_key_iv: "11684a0f8223f8ae38cbb2cdfe9e9f4e"
server_key_version: "encryptionkey@0"
I20260924 01:00:57.066006 27797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:00:57.086661 27797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:00:57.090611 27797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:00:57.090900 27797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:00:57.109871 27797 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.190:38985
I20260924 01:00:57.109915 27863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.190:38985 every 8 connection(s)
I20260924 01:00:57.111261 27797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 01:00:57.114971 27864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:00:57.119196 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 27797
I20260924 01:00:57.119374 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 01:00:57.119959 8450 external_mini_cluster.cc:1428] Setting key 56e196c3d8622a58502976f2ef95ca24
I20260924 01:00:57.126595 27864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b2d4d8c2f9a400998ef07b4860b3bb8: Bootstrap starting.
I20260924 01:00:57.130386 27864 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b2d4d8c2f9a400998ef07b4860b3bb8: Neither blocks nor log segments found. Creating new log.
I20260924 01:00:57.131695 27864 log.cc:824] T 00000000000000000000000000000000 P 0b2d4d8c2f9a400998ef07b4860b3bb8: Log is configured to *not* fsync() on all Append() calls
I20260924 01:00:57.134821 27864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b2d4d8c2f9a400998ef07b4860b3bb8: No bootstrap required, opened a new log
Sep 24 01:00:57 dist-test-slave-mmhs krb5kdc[27784](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 01:00:57.143229 27864 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b2d4d8c2f9a400998ef07b4860b3bb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b2d4d8c2f9a400998ef07b4860b3bb8" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 38985 } }
I20260924 01:00:57.143712 27864 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b2d4d8c2f9a400998ef07b4860b3bb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 01:00:57.143843 27864 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b2d4d8c2f9a400998ef07b4860b3bb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b2d4d8c2f9a400998ef07b4860b3bb8, State: Initialized, Role: FOLLOWER
I20260924 01:00:57.144619 27867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:57.121922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44186 (local address 127.8.64.190:38985)
0924 01:00:57.122739 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:57.122750 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:57.122789 (+ 39us) server_negotiation.cc:407] Connection header received
0924 01:00:57.125146 (+ 2357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:57.125168 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:57.125561 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:57.125885 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:57.127797 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:57.129423 (+ 1626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:57.131534 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:57.132498 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:57.132723 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:57.136709 (+ 3986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:57.136717 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:57.136732 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:00:57.136775 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:57.140208 (+ 3433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:57.140988 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:57.140994 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:57.141001 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:57.141170 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:57.141586 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:57.141594 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:57.141596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:57.142181 (+ 585us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:57.142550 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:57.142967 (+ 417us) server_negotiation.cc:299] Negotiation successful
0924 01:00:57.143399 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":410,"threads_started":1}
I20260924 01:00:57.144500 27864 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b2d4d8c2f9a400998ef07b4860b3bb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b2d4d8c2f9a400998ef07b4860b3bb8" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 38985 } }
I20260924 01:00:57.144802 27864 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b2d4d8c2f9a400998ef07b4860b3bb8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 01:00:57.144984 27864 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b2d4d8c2f9a400998ef07b4860b3bb8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 01:00:57.145183 27864 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b2d4d8c2f9a400998ef07b4860b3bb8 [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:00:57.148102 27864 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b2d4d8c2f9a400998ef07b4860b3bb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b2d4d8c2f9a400998ef07b4860b3bb8" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 38985 } }
I20260924 01:00:57.148541 27864 leader_election.cc:302] T 00000000000000000000000000000000 P 0b2d4d8c2f9a400998ef07b4860b3bb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b2d4d8c2f9a400998ef07b4860b3bb8; no voters:
I20260924 01:00:57.149114 27864 leader_election.cc:288] T 00000000000000000000000000000000 P 0b2d4d8c2f9a400998ef07b4860b3bb8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 01:00:57.149288 27869 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b2d4d8c2f9a400998ef07b4860b3bb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 01:00:57.150425 27869 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b2d4d8c2f9a400998ef07b4860b3bb8 [term 1 LEADER]: Becoming Leader. State: Replica: 0b2d4d8c2f9a400998ef07b4860b3bb8, State: Running, Role: LEADER
I20260924 01:00:57.150959 27869 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b2d4d8c2f9a400998ef07b4860b3bb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b2d4d8c2f9a400998ef07b4860b3bb8" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 38985 } }
I20260924 01:00:57.151260 27864 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b2d4d8c2f9a400998ef07b4860b3bb8 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 01:00:57.154393 27871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b2d4d8c2f9a400998ef07b4860b3bb8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b2d4d8c2f9a400998ef07b4860b3bb8. Latest consensus state: current_term: 1 leader_uuid: "0b2d4d8c2f9a400998ef07b4860b3bb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b2d4d8c2f9a400998ef07b4860b3bb8" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 38985 } } }
I20260924 01:00:57.154855 27871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b2d4d8c2f9a400998ef07b4860b3bb8 [sys.catalog]: This master's current role is: LEADER
I20260924 01:00:57.154393 27870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b2d4d8c2f9a400998ef07b4860b3bb8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b2d4d8c2f9a400998ef07b4860b3bb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b2d4d8c2f9a400998ef07b4860b3bb8" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 38985 } } }
I20260924 01:00:57.155256 27870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b2d4d8c2f9a400998ef07b4860b3bb8 [sys.catalog]: This master's current role is: LEADER
I20260924 01:00:57.156687 27875 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 01:00:57.162074 27875 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 01:00:57.171353 27875 catalog_manager.cc:1347] Generated new cluster ID: b4210b34675a46eea95c124017df9cdb
I20260924 01:00:57.171473 27875 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 01:00:57.186902 27875 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 01:00:57.188421 27875 catalog_manager.cc:1504] Loading token signing keys...
I20260924 01:00:57.204438 27875 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b2d4d8c2f9a400998ef07b4860b3bb8: Generated new TSK 0
I20260924 01:00:57.205287 27875 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.64.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab.
Entry for principal kudu/127.8.64.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab.
I20260924 01:00:57.269989 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.129:0
--local_ip_for_outbound_sockets=127.8.64.129
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:38985
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab
--principal=kudu/127.8.64.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:40193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 01:00:57.605830 27890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:00:57.606108 27890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:00:57.606271 27890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:00:57.613758 27890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:00:57.613868 27890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:00:57.614048 27890 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.129
I20260924 01:00:57.623927 27890 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:40193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.129.keytab
--principal=kudu/127.8.64.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:38985
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.129
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:00:57.625224 27890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:00:57.626627 27890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:00:57.640645 27895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:57.640656 27896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:57.640892 27898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:00:57.641657 27890 server_base.cc:1034] running on GCE node
I20260924 01:00:57.642397 27890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:00:57.643421 27890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:00:57.644701 27890 hybrid_clock.cc:648] HybridClock initialized: now 1790211657644633 us; error 50 us; skew 500 ppm
I20260924 01:00:57.645164 27890 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 01:00:57 dist-test-slave-mmhs krb5kdc[27784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211657, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 01:00:57.648737 27890 init.cc:377] Logged in from keytab as kudu/127.8.64.129@KRBTEST.COM (short username kudu)
I20260924 01:00:57.650283 27890 webserver.cc:460] Webserver started at http://127.8.64.129:38589/ using document root <none> and password file <none>
I20260924 01:00:57.651031 27890 fs_manager.cc:359] Metadata directory not provided
I20260924 01:00:57.651168 27890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:00:57.651554 27890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:00:57.654477 27890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-0/data/instance:
uuid: "aada506276a7447ea035a996041bb122"
format_stamp: "Formatted at 2026-09-24 01:00:57 on dist-test-slave-mmhs"
server_key: "6683c053c6691ad9a8268d841956fddd"
server_key_iv: "185f4dab9ecd8694120d43cb8fb75a40"
server_key_version: "encryptionkey@0"
I20260924 01:00:57.655256 27890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance:
uuid: "aada506276a7447ea035a996041bb122"
format_stamp: "Formatted at 2026-09-24 01:00:57 on dist-test-slave-mmhs"
server_key: "6683c053c6691ad9a8268d841956fddd"
server_key_iv: "185f4dab9ecd8694120d43cb8fb75a40"
server_key_version: "encryptionkey@0"
I20260924 01:00:57.660526 27890 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260924 01:00:57.663995 27905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:57.665339 27890 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260924 01:00:57.665520 27890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-0/wal
uuid: "aada506276a7447ea035a996041bb122"
format_stamp: "Formatted at 2026-09-24 01:00:57 on dist-test-slave-mmhs"
server_key: "6683c053c6691ad9a8268d841956fddd"
server_key_iv: "185f4dab9ecd8694120d43cb8fb75a40"
server_key_version: "encryptionkey@0"
I20260924 01:00:57.665716 27890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:00:57.681221 27890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:00:57.685485 27890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:00:57.685802 27890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:00:57.686780 27890 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 01:00:57.688138 27890 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 01:00:57.688244 27890 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:57.688395 27890 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 01:00:57.688472 27890 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:57.722687 27890 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.129:36855
I20260924 01:00:57.722735 28017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.129:36855 every 8 connection(s)
I20260924 01:00:57.724025 27890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
I20260924 01:00:57.726199 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 27890
I20260924 01:00:57.726428 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance
I20260924 01:00:57.726991 8450 external_mini_cluster.cc:1428] Setting key 4ca9ea79ec4330f3820ca7ae337cd7f7
Sep 24 01:00:57 dist-test-slave-mmhs krb5kdc[27784](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211657, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.129@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 01:00:57.747577 28020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:57.726933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:33263 (local address 127.8.64.190:38985)
0924 01:00:57.730748 (+ 3815us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:57.730754 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:57.730773 (+ 19us) server_negotiation.cc:407] Connection header received
0924 01:00:57.730824 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:57.730833 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:57.730920 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:57.731069 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:57.733058 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:57.733774 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:57.736556 (+ 2782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:57.736976 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:57.737084 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:57.741535 (+ 4451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:57.741541 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:57.741544 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:00:57.741578 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:57.744541 (+ 2963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:57.745269 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:57.745274 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:57.745276 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:57.745397 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:57.745828 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:57.745838 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:57.745840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:57.746097 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:57.746251 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:57.746972 (+ 721us) server_negotiation.cc:299] Negotiation successful
0924 01:00:57.747204 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3678,"thread_start_us":3556,"threads_started":1}
I20260924 01:00:57.748525 28021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:57.727407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:38985 (local address 127.8.64.129:33263)
0924 01:00:57.728604 (+ 1197us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:57.728656 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:57.730448 (+ 1792us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:57.731322 (+ 874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:57.731331 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:57.731995 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0924 01:00:57.732848 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:57.732879 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:57.733957 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:57.733966 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:57.736373 (+ 2407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:57.736394 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:57.737217 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:57.737227 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:57.737579 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:57.738370 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 01:00:57.738406 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0924 01:00:57.741277 (+ 2871us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 01:00:57.744690 (+ 3413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:00:57.744698 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:00:57.744714 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:00:57.745130 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:00:57.745508 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:00:57.745513 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:00:57.745517 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:00:57.745705 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:00:57.746223 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 01:00:57.746231 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 01:00:57.746606 (+ 375us) client_negotiation.cc:769] Sending connection context
0924 01:00:57.746867 (+ 261us) client_negotiation.cc:240] Negotiation successful
0924 01:00:57.747382 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":918,"thread_start_us":683,"threads_started":1}
I20260924 01:00:57.750154 28018 heartbeater.cc:347] Connected to a master server at 127.8.64.190:38985
I20260924 01:00:57.750494 28018 heartbeater.cc:464] Registering TS with master...
I20260924 01:00:57.751252 28018 heartbeater.cc:511] Master 127.8.64.190:38985 requested a full tablet report, sending...
I20260924 01:00:57.753695 27829 ts_manager.cc:194] Registered new tserver with Master: aada506276a7447ea035a996041bb122 (127.8.64.129:36855)
I20260924 01:00:57.755976 27829 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.129@KRBTEST.COM'} at 127.8.64.129:33263
WARNING: no policy specified for kudu/127.8.64.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab.
Entry for principal kudu/127.8.64.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab.
I20260924 01:00:57.794245 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.130:0
--local_ip_for_outbound_sockets=127.8.64.130
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:38985
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab
--principal=kudu/127.8.64.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:40193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 01:00:58.124313 28024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:00:58.124576 28024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:00:58.124737 28024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:00:58.132184 28024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:00:58.132292 28024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:00:58.132467 28024 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.130
I20260924 01:00:58.142561 28024 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:40193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.130.keytab
--principal=kudu/127.8.64.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:38985
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.130
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:00:58.143931 28024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:00:58.145249 28024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:00:58.159399 28029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:58.159639 28030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:58.159646 28032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:00:58.160915 28024 server_base.cc:1034] running on GCE node
I20260924 01:00:58.161626 28024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:00:58.162691 28024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:00:58.163930 28024 hybrid_clock.cc:648] HybridClock initialized: now 1790211658163873 us; error 71 us; skew 500 ppm
I20260924 01:00:58.164258 28024 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 01:00:58 dist-test-slave-mmhs krb5kdc[27784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211658, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 01:00:58.171322 28024 init.cc:377] Logged in from keytab as kudu/127.8.64.130@KRBTEST.COM (short username kudu)
I20260924 01:00:58.172857 28024 webserver.cc:460] Webserver started at http://127.8.64.130:38065/ using document root <none> and password file <none>
I20260924 01:00:58.173537 28024 fs_manager.cc:359] Metadata directory not provided
I20260924 01:00:58.173638 28024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:00:58.173898 28024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:00:58.176880 28024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-1/data/instance:
uuid: "f394707d72a74728bb8540687fd8fa28"
format_stamp: "Formatted at 2026-09-24 01:00:58 on dist-test-slave-mmhs"
server_key: "e98cfeecf5196a4e09067306f7f0e322"
server_key_iv: "fa697e2e9cf521c9fd14ac14fc72a346"
server_key_version: "encryptionkey@0"
I20260924 01:00:58.177620 28024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance:
uuid: "f394707d72a74728bb8540687fd8fa28"
format_stamp: "Formatted at 2026-09-24 01:00:58 on dist-test-slave-mmhs"
server_key: "e98cfeecf5196a4e09067306f7f0e322"
server_key_iv: "fa697e2e9cf521c9fd14ac14fc72a346"
server_key_version: "encryptionkey@0"
I20260924 01:00:58.182968 28024 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260924 01:00:58.186501 28039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:58.188014 28024 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260924 01:00:58.188232 28024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-1/wal
uuid: "f394707d72a74728bb8540687fd8fa28"
format_stamp: "Formatted at 2026-09-24 01:00:58 on dist-test-slave-mmhs"
server_key: "e98cfeecf5196a4e09067306f7f0e322"
server_key_iv: "fa697e2e9cf521c9fd14ac14fc72a346"
server_key_version: "encryptionkey@0"
I20260924 01:00:58.188457 28024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:00:58.202430 28024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:00:58.208002 28024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:00:58.208367 28024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:00:58.209723 28024 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 01:00:58.211712 28024 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 01:00:58.211818 28024 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:58.211949 28024 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 01:00:58.212023 28024 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:58.246608 28024 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.130:37723
I20260924 01:00:58.246624 28151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.130:37723 every 8 connection(s)
I20260924 01:00:58.247988 28024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
I20260924 01:00:58.252624 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 28024
I20260924 01:00:58.252811 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance
I20260924 01:00:58.253345 8450 external_mini_cluster.cc:1428] Setting key c3a6d4c6df334064232c592cdddac908
Sep 24 01:00:58 dist-test-slave-mmhs krb5kdc[27784](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211658, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.130@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 01:00:58.271231 28154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:58.250945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:40353 (local address 127.8.64.190:38985)
0924 01:00:58.251522 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:58.251527 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:58.252668 (+ 1141us) server_negotiation.cc:407] Connection header received
0924 01:00:58.255009 (+ 2341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:58.255013 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:58.255078 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:58.255227 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:58.257086 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:58.257997 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:58.259745 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:58.260228 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:58.260374 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:58.264172 (+ 3798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:58.264177 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:58.264181 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:00:58.264218 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:58.267258 (+ 3040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:58.268099 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:58.268104 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:58.268107 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:58.268244 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:58.269004 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:58.269009 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:58.269011 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:58.269465 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:58.269611 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:58.270712 (+ 1101us) server_negotiation.cc:299] Negotiation successful
0924 01:00:58.270948 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":267,"threads_started":1}
I20260924 01:00:58.272748 28155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:58.251524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:38985 (local address 127.8.64.130:40353)
0924 01:00:58.252486 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:58.252547 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:58.254672 (+ 2125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:58.255565 (+ 893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:58.255575 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:58.256158 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0924 01:00:58.256910 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:58.256930 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:58.258165 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:58.258169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:58.259591 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:58.259603 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:58.260489 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:58.260496 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:58.260732 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:58.261486 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 01:00:58.261511 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0924 01:00:58.263910 (+ 2399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 01:00:58.267434 (+ 3524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:00:58.267444 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:00:58.267465 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:00:58.267944 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:00:58.268442 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:00:58.268448 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:00:58.268455 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:00:58.268764 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:00:58.269626 (+ 862us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 01:00:58.269637 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 01:00:58.270135 (+ 498us) client_negotiation.cc:769] Sending connection context
0924 01:00:58.270564 (+ 429us) client_negotiation.cc:240] Negotiation successful
0924 01:00:58.271204 (+ 640us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":314,"threads_started":1}
I20260924 01:00:58.274395 28152 heartbeater.cc:347] Connected to a master server at 127.8.64.190:38985
I20260924 01:00:58.274746 28152 heartbeater.cc:464] Registering TS with master...
I20260924 01:00:58.275497 28152 heartbeater.cc:511] Master 127.8.64.190:38985 requested a full tablet report, sending...
I20260924 01:00:58.276924 27829 ts_manager.cc:194] Registered new tserver with Master: f394707d72a74728bb8540687fd8fa28 (127.8.64.130:37723)
I20260924 01:00:58.277748 27829 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.130@KRBTEST.COM'} at 127.8.64.130:40353
WARNING: no policy specified for kudu/127.8.64.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab.
Entry for principal kudu/127.8.64.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab.
I20260924 01:00:58.314404 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.131:0
--local_ip_for_outbound_sockets=127.8.64.131
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:38985
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab
--principal=kudu/127.8.64.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:40193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 01:00:58.647631 28158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:00:58.647933 28158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:00:58.648044 28158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:00:58.655778 28158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:00:58.655898 28158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:00:58.656015 28158 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.131
I20260924 01:00:58.666656 28158 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:40193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.131.keytab
--principal=kudu/127.8.64.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:38985
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.131
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:00:58.667883 28158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:00:58.669402 28158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:00:58.682631 28164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:58.682729 28163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:00:58.682729 28166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:00:58.683642 28158 server_base.cc:1034] running on GCE node
I20260924 01:00:58.684191 28158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:00:58.685130 28158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:00:58.686398 28158 hybrid_clock.cc:648] HybridClock initialized: now 1790211658686341 us; error 61 us; skew 500 ppm
I20260924 01:00:58.686719 28158 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 01:00:58 dist-test-slave-mmhs krb5kdc[27784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211658, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 01:00:58.689811 28158 init.cc:377] Logged in from keytab as kudu/127.8.64.131@KRBTEST.COM (short username kudu)
I20260924 01:00:58.691313 28158 webserver.cc:460] Webserver started at http://127.8.64.131:36369/ using document root <none> and password file <none>
I20260924 01:00:58.692044 28158 fs_manager.cc:359] Metadata directory not provided
I20260924 01:00:58.692186 28158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:00:58.692498 28158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:00:58.695408 28158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-2/data/instance:
uuid: "5ee2fe427ce344ec92a057fc68336b35"
format_stamp: "Formatted at 2026-09-24 01:00:58 on dist-test-slave-mmhs"
server_key: "3539989ef0f5519610ba62bff3fd93d9"
server_key_iv: "b8e3da98d2c7c5ca8078242faf94eb29"
server_key_version: "encryptionkey@0"
I20260924 01:00:58.696193 28158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance:
uuid: "5ee2fe427ce344ec92a057fc68336b35"
format_stamp: "Formatted at 2026-09-24 01:00:58 on dist-test-slave-mmhs"
server_key: "3539989ef0f5519610ba62bff3fd93d9"
server_key_iv: "b8e3da98d2c7c5ca8078242faf94eb29"
server_key_version: "encryptionkey@0"
I20260924 01:00:58.701175 28158 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260924 01:00:58.704618 28173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:58.705963 28158 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260924 01:00:58.706153 28158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-2/wal
uuid: "5ee2fe427ce344ec92a057fc68336b35"
format_stamp: "Formatted at 2026-09-24 01:00:58 on dist-test-slave-mmhs"
server_key: "3539989ef0f5519610ba62bff3fd93d9"
server_key_iv: "b8e3da98d2c7c5ca8078242faf94eb29"
server_key_version: "encryptionkey@0"
I20260924 01:00:58.706416 28158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:00:58.725843 28158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:00:58.730851 28158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:00:58.731091 28158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:00:58.731972 28158 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 01:00:58.733232 28158 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 01:00:58.733305 28158 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:58.733390 28158 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 01:00:58.733443 28158 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 01:00:58.759514 28018 heartbeater.cc:503] Master 127.8.64.190:38985 was elected leader, sending a full tablet report...
I20260924 01:00:58.769495 28158 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.131:33049
I20260924 01:00:58.769560 28285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.131:33049 every 8 connection(s)
I20260924 01:00:58.771098 28158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
I20260924 01:00:58.780915 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 28158
I20260924 01:00:58.781140 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance
I20260924 01:00:58.781579 8450 external_mini_cluster.cc:1428] Setting key 1f13b2b4dadf7bbc3a904895d9d7b9f3
Sep 24 01:00:58 dist-test-slave-mmhs krb5kdc[27784](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211658, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.131@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 01:00:58.791852 28288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:58.773783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:59791 (local address 127.8.64.190:38985)
0924 01:00:58.774282 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:58.774347 (+ 65us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:58.775470 (+ 1123us) server_negotiation.cc:407] Connection header received
0924 01:00:58.777350 (+ 1880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:58.777357 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:58.777463 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:58.777597 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:58.779167 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:58.780058 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:58.781710 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:58.782243 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:58.782401 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:58.786202 (+ 3801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:58.786207 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:58.786210 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:00:58.786243 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:58.788746 (+ 2503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:58.789483 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:58.789488 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:58.789490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:58.789636 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:58.790098 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:58.790102 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:58.790105 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:58.790480 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:58.790644 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:58.791397 (+ 753us) server_negotiation.cc:299] Negotiation successful
0924 01:00:58.791578 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":255,"threads_started":1}
I20260924 01:00:58.792903 28289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:58.774500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:38985 (local address 127.8.64.131:59791)
0924 01:00:58.775253 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:58.775298 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:58.777075 (+ 1777us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:58.777839 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:58.777847 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:58.778435 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0924 01:00:58.778997 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:58.779014 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:58.780222 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:58.780227 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:58.781560 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:58.781571 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:58.782506 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:58.782515 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:58.782820 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:58.783618 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 01:00:58.783659 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0924 01:00:58.785970 (+ 2311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 01:00:58.788877 (+ 2907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:00:58.788884 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:00:58.788897 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:00:58.789322 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:00:58.789764 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:00:58.789768 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:00:58.789771 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:00:58.789958 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:00:58.790656 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 01:00:58.790663 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 01:00:58.791038 (+ 375us) client_negotiation.cc:769] Sending connection context
0924 01:00:58.791289 (+ 251us) client_negotiation.cc:240] Negotiation successful
0924 01:00:58.791684 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":317,"threads_started":1}
I20260924 01:00:58.794543 28286 heartbeater.cc:347] Connected to a master server at 127.8.64.190:38985
I20260924 01:00:58.794888 28286 heartbeater.cc:464] Registering TS with master...
I20260924 01:00:58.795720 28286 heartbeater.cc:511] Master 127.8.64.190:38985 requested a full tablet report, sending...
I20260924 01:00:58.797375 27829 ts_manager.cc:194] Registered new tserver with Master: 5ee2fe427ce344ec92a057fc68336b35 (127.8.64.131:33049)
I20260924 01:00:58.798226 27829 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.64.131@KRBTEST.COM'} at 127.8.64.131:59791
I20260924 01:00:58.809732 8450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 01:00:58.828035 28288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:58.814352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44196 (local address 127.8.64.190:38985)
0924 01:00:58.814624 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:58.814633 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:58.814994 (+ 361us) server_negotiation.cc:407] Connection header received
0924 01:00:58.815216 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:58.815222 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:58.815308 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:58.815423 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:58.816987 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:58.817759 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:58.819318 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:58.819754 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:58.819924 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:58.822600 (+ 2676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:58.822606 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:58.822610 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:00:58.822642 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:58.825213 (+ 2571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:58.825916 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:58.825922 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:58.825926 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:58.826065 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:58.826523 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:58.826528 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:58.826532 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:58.826852 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:58.827109 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:58.827465 (+ 356us) server_negotiation.cc:299] Negotiation successful
0924 01:00:58.827710 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260924 01:00:58.832231 27829 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44196:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 01:00:58.839879 27829 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 01:00:58.878845 28300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:58.852426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49180 (local address 127.8.64.129:36855)
0924 01:00:58.870400 (+ 17974us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:58.870409 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:58.870431 (+ 22us) server_negotiation.cc:407] Connection header received
0924 01:00:58.870512 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:58.870518 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:58.870694 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:58.871315 (+ 621us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 01:00:58.872072 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:58.873598 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:58.876379 (+ 2781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:58.877652 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:58.877816 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:58.878063 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:58.878317 (+ 254us) server_negotiation.cc:299] Negotiation successful
0924 01:00:58.878489 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17748,"thread_start_us":17677,"threads_started":1}
I20260924 01:00:58.878947 28302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:58.852165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:36855 (local address 127.0.0.1:49180)
0924 01:00:58.862043 (+ 9878us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:58.862100 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:58.862348 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:58.870901 (+ 8553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:58.870905 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:58.870937 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 01:00:58.871848 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:58.871865 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:58.873775 (+ 1910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:58.873780 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:58.876185 (+ 2405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:58.876202 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:58.877855 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:58.877861 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:58.878047 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:58.878103 (+ 56us) client_negotiation.cc:769] Sending connection context
0924 01:00:58.878257 (+ 154us) client_negotiation.cc:240] Negotiation successful
0924 01:00:58.878509 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9548,"mutex_wait_us":311,"thread_start_us":16171,"threads_started":1}
I20260924 01:00:58.879868 28299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:58.852935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.131:33049 (local address 127.0.0.1:54038)
0924 01:00:58.869039 (+ 16104us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:58.869064 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:58.869287 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:58.870782 (+ 1495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:58.870789 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:58.870819 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 01:00:58.871499 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:58.871518 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:58.873615 (+ 2097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:58.873619 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:58.876529 (+ 2910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:58.876537 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:58.878703 (+ 2166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:58.878709 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:58.878922 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:58.878963 (+ 41us) client_negotiation.cc:769] Sending connection context
0924 01:00:58.879075 (+ 112us) client_negotiation.cc:240] Negotiation successful
0924 01:00:58.879211 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15955,"mutex_wait_us":309,"thread_start_us":8596,"threads_started":1}
I20260924 01:00:58.879957 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:58.853598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54038 (local address 127.8.64.131:33049)
0924 01:00:58.869978 (+ 16380us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:58.869988 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:58.870022 (+ 34us) server_negotiation.cc:407] Connection header received
0924 01:00:58.870127 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:58.870137 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:58.870567 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:58.870779 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 01:00:58.871745 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:58.873442 (+ 1697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:58.876719 (+ 3277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:58.878158 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:58.878280 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:58.878515 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:58.879451 (+ 936us) server_negotiation.cc:299] Negotiation successful
0924 01:00:58.879618 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16180,"thread_start_us":16014,"threads_started":1}
I20260924 01:00:58.880038 28303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:58.853187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45434 (local address 127.8.64.130:37723)
0924 01:00:58.870255 (+ 17068us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:58.870264 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:58.870313 (+ 49us) server_negotiation.cc:407] Connection header received
0924 01:00:58.870419 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:58.870428 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:58.870669 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:58.870927 (+ 258us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 01:00:58.872099 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:58.873405 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:58.876706 (+ 3301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:58.878147 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:58.878280 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:58.878482 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:58.879446 (+ 964us) server_negotiation.cc:299] Negotiation successful
0924 01:00:58.879728 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16900,"thread_start_us":16715,"threads_started":1}
I20260924 01:00:58.880642 28301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:58.852603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.130:37723 (local address 127.0.0.1:45434)
0924 01:00:58.868349 (+ 15746us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:58.868379 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:58.868544 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:58.871081 (+ 2537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:58.871088 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:58.871117 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 01:00:58.871868 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:58.871883 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:58.873545 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:58.873553 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:58.876522 (+ 2969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:58.876537 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:58.878684 (+ 2147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:58.878692 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:58.879124 (+ 432us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:58.879155 (+ 31us) client_negotiation.cc:769] Sending connection context
0924 01:00:58.879224 (+ 69us) client_negotiation.cc:240] Negotiation successful
0924 01:00:58.879344 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15565,"mutex_wait_us":269,"thread_start_us":15368,"threads_started":1}
I20260924 01:00:58.883832 28087 tablet_service.cc:1467] Processing CreateTablet for tablet 13576d3b2a804c9e9fc3c94169754daf (DEFAULT_TABLE table=test-table [id=6cca1d65a2a6425d897f56174d2bd1f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 01:00:58.883647 28221 tablet_service.cc:1467] Processing CreateTablet for tablet 13576d3b2a804c9e9fc3c94169754daf (DEFAULT_TABLE table=test-table [id=6cca1d65a2a6425d897f56174d2bd1f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 01:00:58.883647 27953 tablet_service.cc:1467] Processing CreateTablet for tablet 13576d3b2a804c9e9fc3c94169754daf (DEFAULT_TABLE table=test-table [id=6cca1d65a2a6425d897f56174d2bd1f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 01:00:58.884969 28087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13576d3b2a804c9e9fc3c94169754daf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:00:58.885099 28221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13576d3b2a804c9e9fc3c94169754daf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:00:58.885110 27953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13576d3b2a804c9e9fc3c94169754daf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:00:58.896799 28305 tablet_bootstrap.cc:492] T 13576d3b2a804c9e9fc3c94169754daf P f394707d72a74728bb8540687fd8fa28: Bootstrap starting.
I20260924 01:00:58.900317 28305 tablet_bootstrap.cc:654] T 13576d3b2a804c9e9fc3c94169754daf P f394707d72a74728bb8540687fd8fa28: Neither blocks nor log segments found. Creating new log.
I20260924 01:00:58.901343 28305 log.cc:824] T 13576d3b2a804c9e9fc3c94169754daf P f394707d72a74728bb8540687fd8fa28: Log is configured to *not* fsync() on all Append() calls
I20260924 01:00:58.904281 28305 tablet_bootstrap.cc:492] T 13576d3b2a804c9e9fc3c94169754daf P f394707d72a74728bb8540687fd8fa28: No bootstrap required, opened a new log
I20260924 01:00:58.904554 28305 ts_tablet_manager.cc:1397] T 13576d3b2a804c9e9fc3c94169754daf P f394707d72a74728bb8540687fd8fa28: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260924 01:00:58.906769 28306 tablet_bootstrap.cc:492] T 13576d3b2a804c9e9fc3c94169754daf P aada506276a7447ea035a996041bb122: Bootstrap starting.
I20260924 01:00:58.907326 28307 tablet_bootstrap.cc:492] T 13576d3b2a804c9e9fc3c94169754daf P 5ee2fe427ce344ec92a057fc68336b35: Bootstrap starting.
I20260924 01:00:58.910601 28306 tablet_bootstrap.cc:654] T 13576d3b2a804c9e9fc3c94169754daf P aada506276a7447ea035a996041bb122: Neither blocks nor log segments found. Creating new log.
I20260924 01:00:58.910611 28307 tablet_bootstrap.cc:654] T 13576d3b2a804c9e9fc3c94169754daf P 5ee2fe427ce344ec92a057fc68336b35: Neither blocks nor log segments found. Creating new log.
I20260924 01:00:58.911832 28307 log.cc:824] T 13576d3b2a804c9e9fc3c94169754daf P 5ee2fe427ce344ec92a057fc68336b35: Log is configured to *not* fsync() on all Append() calls
I20260924 01:00:58.911844 28306 log.cc:824] T 13576d3b2a804c9e9fc3c94169754daf P aada506276a7447ea035a996041bb122: Log is configured to *not* fsync() on all Append() calls
I20260924 01:00:58.914135 28305 raft_consensus.cc:348] T 13576d3b2a804c9e9fc3c94169754daf P f394707d72a74728bb8540687fd8fa28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aada506276a7447ea035a996041bb122" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 36855 } } peers { permanent_uuid: "f394707d72a74728bb8540687fd8fa28" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 37723 } } peers { permanent_uuid: "5ee2fe427ce344ec92a057fc68336b35" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 33049 } }
I20260924 01:00:58.914619 28305 raft_consensus.cc:374] T 13576d3b2a804c9e9fc3c94169754daf P f394707d72a74728bb8540687fd8fa28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 01:00:58.914726 28305 raft_consensus.cc:729] T 13576d3b2a804c9e9fc3c94169754daf P f394707d72a74728bb8540687fd8fa28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f394707d72a74728bb8540687fd8fa28, State: Initialized, Role: FOLLOWER
I20260924 01:00:58.915431 28306 tablet_bootstrap.cc:492] T 13576d3b2a804c9e9fc3c94169754daf P aada506276a7447ea035a996041bb122: No bootstrap required, opened a new log
I20260924 01:00:58.915360 28305 consensus_queue.cc:260] T 13576d3b2a804c9e9fc3c94169754daf P f394707d72a74728bb8540687fd8fa28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aada506276a7447ea035a996041bb122" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 36855 } } peers { permanent_uuid: "f394707d72a74728bb8540687fd8fa28" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 37723 } } peers { permanent_uuid: "5ee2fe427ce344ec92a057fc68336b35" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 33049 } }
I20260924 01:00:58.915705 28306 ts_tablet_manager.cc:1397] T 13576d3b2a804c9e9fc3c94169754daf P aada506276a7447ea035a996041bb122: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260924 01:00:58.915896 28307 tablet_bootstrap.cc:492] T 13576d3b2a804c9e9fc3c94169754daf P 5ee2fe427ce344ec92a057fc68336b35: No bootstrap required, opened a new log
I20260924 01:00:58.916162 28307 ts_tablet_manager.cc:1397] T 13576d3b2a804c9e9fc3c94169754daf P 5ee2fe427ce344ec92a057fc68336b35: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260924 01:00:58.923163 28152 heartbeater.cc:503] Master 127.8.64.190:38985 was elected leader, sending a full tablet report...
I20260924 01:00:58.923907 28305 ts_tablet_manager.cc:1428] T 13576d3b2a804c9e9fc3c94169754daf P f394707d72a74728bb8540687fd8fa28: Time spent starting tablet: real 0.019s user 0.008s sys 0.011s
I20260924 01:00:58.924015 28307 raft_consensus.cc:348] T 13576d3b2a804c9e9fc3c94169754daf P 5ee2fe427ce344ec92a057fc68336b35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aada506276a7447ea035a996041bb122" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 36855 } } peers { permanent_uuid: "f394707d72a74728bb8540687fd8fa28" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 37723 } } peers { permanent_uuid: "5ee2fe427ce344ec92a057fc68336b35" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 33049 } }
I20260924 01:00:58.924505 28307 raft_consensus.cc:374] T 13576d3b2a804c9e9fc3c94169754daf P 5ee2fe427ce344ec92a057fc68336b35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 01:00:58.924619 28307 raft_consensus.cc:729] T 13576d3b2a804c9e9fc3c94169754daf P 5ee2fe427ce344ec92a057fc68336b35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ee2fe427ce344ec92a057fc68336b35, State: Initialized, Role: FOLLOWER
I20260924 01:00:58.925292 28307 consensus_queue.cc:260] T 13576d3b2a804c9e9fc3c94169754daf P 5ee2fe427ce344ec92a057fc68336b35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aada506276a7447ea035a996041bb122" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 36855 } } peers { permanent_uuid: "f394707d72a74728bb8540687fd8fa28" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 37723 } } peers { permanent_uuid: "5ee2fe427ce344ec92a057fc68336b35" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 33049 } }
I20260924 01:00:58.926532 28286 heartbeater.cc:503] Master 127.8.64.190:38985 was elected leader, sending a full tablet report...
I20260924 01:00:58.927106 28307 ts_tablet_manager.cc:1428] T 13576d3b2a804c9e9fc3c94169754daf P 5ee2fe427ce344ec92a057fc68336b35: Time spent starting tablet: real 0.011s user 0.001s sys 0.009s
I20260924 01:00:58.929463 28306 raft_consensus.cc:348] T 13576d3b2a804c9e9fc3c94169754daf P aada506276a7447ea035a996041bb122 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aada506276a7447ea035a996041bb122" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 36855 } } peers { permanent_uuid: "f394707d72a74728bb8540687fd8fa28" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 37723 } } peers { permanent_uuid: "5ee2fe427ce344ec92a057fc68336b35" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 33049 } }
I20260924 01:00:58.929932 28306 raft_consensus.cc:374] T 13576d3b2a804c9e9fc3c94169754daf P aada506276a7447ea035a996041bb122 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 01:00:58.930037 28306 raft_consensus.cc:729] T 13576d3b2a804c9e9fc3c94169754daf P aada506276a7447ea035a996041bb122 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aada506276a7447ea035a996041bb122, State: Initialized, Role: FOLLOWER
I20260924 01:00:58.930666 28306 consensus_queue.cc:260] T 13576d3b2a804c9e9fc3c94169754daf P aada506276a7447ea035a996041bb122 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aada506276a7447ea035a996041bb122" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 36855 } } peers { permanent_uuid: "f394707d72a74728bb8540687fd8fa28" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 37723 } } peers { permanent_uuid: "5ee2fe427ce344ec92a057fc68336b35" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 33049 } }
I20260924 01:00:58.932504 28306 ts_tablet_manager.cc:1428] T 13576d3b2a804c9e9fc3c94169754daf P aada506276a7447ea035a996041bb122: Time spent starting tablet: real 0.017s user 0.012s sys 0.003s
W20260924 01:00:58.976570 28019 tablet.cc:2369] T 13576d3b2a804c9e9fc3c94169754daf P aada506276a7447ea035a996041bb122: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 01:00:59.000139 28153 tablet.cc:2369] T 13576d3b2a804c9e9fc3c94169754daf P f394707d72a74728bb8540687fd8fa28: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 01:00:59.022848 28287 tablet.cc:2369] T 13576d3b2a804c9e9fc3c94169754daf P 5ee2fe427ce344ec92a057fc68336b35: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 01:00:59.032811 28312 raft_consensus.cc:482] T 13576d3b2a804c9e9fc3c94169754daf P 5ee2fe427ce344ec92a057fc68336b35 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 01:00:59.033182 28312 raft_consensus.cc:504] T 13576d3b2a804c9e9fc3c94169754daf P 5ee2fe427ce344ec92a057fc68336b35 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aada506276a7447ea035a996041bb122" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 36855 } } peers { permanent_uuid: "f394707d72a74728bb8540687fd8fa28" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 37723 } } peers { permanent_uuid: "5ee2fe427ce344ec92a057fc68336b35" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 33049 } }
I20260924 01:00:59.035178 28312 leader_election.cc:288] T 13576d3b2a804c9e9fc3c94169754daf P 5ee2fe427ce344ec92a057fc68336b35 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aada506276a7447ea035a996041bb122 (127.8.64.129:36855), f394707d72a74728bb8540687fd8fa28 (127.8.64.130:37723)
I20260924 01:00:59.041492 28300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:59.035627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:35411 (local address 127.8.64.129:36855)
0924 01:00:59.035964 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:59.035971 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:59.035996 (+ 25us) server_negotiation.cc:407] Connection header received
0924 01:00:59.036256 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:59.036264 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:59.036368 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:59.036487 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 01:00:59.036952 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:59.038013 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:59.039829 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:59.040593 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:59.040759 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:59.040835 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:59.041080 (+ 245us) server_negotiation.cc:299] Negotiation successful
0924 01:00:59.041206 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180}
I20260924 01:00:59.041509 28289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:59.035576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:36855 (local address 127.8.64.131:35411)
0924 01:00:59.035881 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:59.035903 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:59.036053 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:59.036521 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:59.036524 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:59.036547 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 01:00:59.036784 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:59.036798 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:59.038144 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:59.038147 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:59.039678 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:59.039689 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:59.040745 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:59.040750 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:59.040928 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:59.040951 (+ 23us) client_negotiation.cc:769] Sending connection context
0924 01:00:59.041056 (+ 105us) client_negotiation.cc:240] Negotiation successful
0924 01:00:59.041185 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126}
I20260924 01:00:59.042174 28303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:59.035654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:41635 (local address 127.8.64.130:37723)
0924 01:00:59.035963 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:59.035970 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:59.036639 (+ 669us) server_negotiation.cc:407] Connection header received
0924 01:00:59.036912 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:59.036918 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:59.037003 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:59.037118 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 01:00:59.037563 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:59.038534 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:59.040302 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:59.041095 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:59.041295 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:59.041388 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:59.041726 (+ 338us) server_negotiation.cc:299] Negotiation successful
0924 01:00:59.041889 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143}
I20260924 01:00:59.042765 28314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:59.035618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.130:37723 (local address 127.8.64.131:41635)
0924 01:00:59.036515 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0924 01:00:59.036533 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0924 01:00:59.036749 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:00:59.037137 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:00:59.037140 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:00:59.037158 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 01:00:59.037405 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:59.037412 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:59.038690 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:59.038698 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:59.040120 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:00:59.040132 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:00:59.041311 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:00:59.041318 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:00:59.041503 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:59.041527 (+ 24us) client_negotiation.cc:769] Sending connection context
0924 01:00:59.042405 (+ 878us) client_negotiation.cc:240] Negotiation successful
0924 01:00:59.042530 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":757,"thread_start_us":549,"threads_started":1}
I20260924 01:00:59.042711 27973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13576d3b2a804c9e9fc3c94169754daf" candidate_uuid: "5ee2fe427ce344ec92a057fc68336b35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aada506276a7447ea035a996041bb122" is_pre_election: true
I20260924 01:00:59.043285 27973 raft_consensus.cc:2454] T 13576d3b2a804c9e9fc3c94169754daf P aada506276a7447ea035a996041bb122 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ee2fe427ce344ec92a057fc68336b35 in term 0.
I20260924 01:00:59.043990 28174 leader_election.cc:302] T 13576d3b2a804c9e9fc3c94169754daf P 5ee2fe427ce344ec92a057fc68336b35 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ee2fe427ce344ec92a057fc68336b35, aada506276a7447ea035a996041bb122; no voters:
I20260924 01:00:59.043949 28107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13576d3b2a804c9e9fc3c94169754daf" candidate_uuid: "5ee2fe427ce344ec92a057fc68336b35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f394707d72a74728bb8540687fd8fa28" is_pre_election: true
I20260924 01:00:59.044379 28312 raft_consensus.cc:2781] T 13576d3b2a804c9e9fc3c94169754daf P 5ee2fe427ce344ec92a057fc68336b35 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 01:00:59.044499 28312 raft_consensus.cc:482] T 13576d3b2a804c9e9fc3c94169754daf P 5ee2fe427ce344ec92a057fc68336b35 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 01:00:59.044486 28107 raft_consensus.cc:2454] T 13576d3b2a804c9e9fc3c94169754daf P f394707d72a74728bb8540687fd8fa28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ee2fe427ce344ec92a057fc68336b35 in term 0.
I20260924 01:00:59.044616 28312 raft_consensus.cc:3037] T 13576d3b2a804c9e9fc3c94169754daf P 5ee2fe427ce344ec92a057fc68336b35 [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:00:59.046873 28312 raft_consensus.cc:504] T 13576d3b2a804c9e9fc3c94169754daf P 5ee2fe427ce344ec92a057fc68336b35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aada506276a7447ea035a996041bb122" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 36855 } } peers { permanent_uuid: "f394707d72a74728bb8540687fd8fa28" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 37723 } } peers { permanent_uuid: "5ee2fe427ce344ec92a057fc68336b35" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 33049 } }
I20260924 01:00:59.047535 28312 leader_election.cc:288] T 13576d3b2a804c9e9fc3c94169754daf P 5ee2fe427ce344ec92a057fc68336b35 [CANDIDATE]: Term 1 election: Requested vote from peers aada506276a7447ea035a996041bb122 (127.8.64.129:36855), f394707d72a74728bb8540687fd8fa28 (127.8.64.130:37723)
I20260924 01:00:59.047982 27973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13576d3b2a804c9e9fc3c94169754daf" candidate_uuid: "5ee2fe427ce344ec92a057fc68336b35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aada506276a7447ea035a996041bb122"
I20260924 01:00:59.048079 28107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13576d3b2a804c9e9fc3c94169754daf" candidate_uuid: "5ee2fe427ce344ec92a057fc68336b35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f394707d72a74728bb8540687fd8fa28"
I20260924 01:00:59.048197 27973 raft_consensus.cc:3037] T 13576d3b2a804c9e9fc3c94169754daf P aada506276a7447ea035a996041bb122 [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:00:59.048328 28107 raft_consensus.cc:3037] T 13576d3b2a804c9e9fc3c94169754daf P f394707d72a74728bb8540687fd8fa28 [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:00:59.050443 27973 raft_consensus.cc:2454] T 13576d3b2a804c9e9fc3c94169754daf P aada506276a7447ea035a996041bb122 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ee2fe427ce344ec92a057fc68336b35 in term 1.
I20260924 01:00:59.050540 28107 raft_consensus.cc:2454] T 13576d3b2a804c9e9fc3c94169754daf P f394707d72a74728bb8540687fd8fa28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ee2fe427ce344ec92a057fc68336b35 in term 1.
I20260924 01:00:59.050833 28174 leader_election.cc:302] T 13576d3b2a804c9e9fc3c94169754daf P 5ee2fe427ce344ec92a057fc68336b35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ee2fe427ce344ec92a057fc68336b35, aada506276a7447ea035a996041bb122; no voters:
I20260924 01:00:59.051136 28312 raft_consensus.cc:2781] T 13576d3b2a804c9e9fc3c94169754daf P 5ee2fe427ce344ec92a057fc68336b35 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 01:00:59.051592 28312 raft_consensus.cc:686] T 13576d3b2a804c9e9fc3c94169754daf P 5ee2fe427ce344ec92a057fc68336b35 [term 1 LEADER]: Becoming Leader. State: Replica: 5ee2fe427ce344ec92a057fc68336b35, State: Running, Role: LEADER
I20260924 01:00:59.052040 28312 consensus_queue.cc:237] T 13576d3b2a804c9e9fc3c94169754daf P 5ee2fe427ce344ec92a057fc68336b35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aada506276a7447ea035a996041bb122" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 36855 } } peers { permanent_uuid: "f394707d72a74728bb8540687fd8fa28" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 37723 } } peers { permanent_uuid: "5ee2fe427ce344ec92a057fc68336b35" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 33049 } }
I20260924 01:00:59.056995 27828 catalog_manager.cc:5495] T 13576d3b2a804c9e9fc3c94169754daf P 5ee2fe427ce344ec92a057fc68336b35 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ee2fe427ce344ec92a057fc68336b35 (127.8.64.131). New cstate: current_term: 1 leader_uuid: "5ee2fe427ce344ec92a057fc68336b35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aada506276a7447ea035a996041bb122" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 36855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f394707d72a74728bb8540687fd8fa28" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 37723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ee2fe427ce344ec92a057fc68336b35" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 33049 } health_report { overall_health: HEALTHY } } }
I20260924 01:00:59.118434 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:59.112693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54048 (local address 127.8.64.131:33049)
0924 01:00:59.112894 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:59.112901 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:59.113063 (+ 162us) server_negotiation.cc:407] Connection header received
0924 01:00:59.113520 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:59.113525 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:59.113618 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:59.113778 (+ 160us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 01:00:59.114163 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:59.114907 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:59.116286 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:59.116695 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:59.116830 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:59.117307 (+ 477us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 01:00:59.117814 (+ 507us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 01:00:59.117982 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:59.118037 (+ 55us) server_negotiation.cc:299] Negotiation successful
0924 01:00:59.118120 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260924 01:00:59.126163 28107 raft_consensus.cc:1261] T 13576d3b2a804c9e9fc3c94169754daf P f394707d72a74728bb8540687fd8fa28 [term 1 FOLLOWER]: Refusing update from remote peer 5ee2fe427ce344ec92a057fc68336b35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 01:00:59.126614 27973 raft_consensus.cc:1261] T 13576d3b2a804c9e9fc3c94169754daf P aada506276a7447ea035a996041bb122 [term 1 FOLLOWER]: Refusing update from remote peer 5ee2fe427ce344ec92a057fc68336b35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 01:00:59.127126 28315 consensus_queue.cc:1035] T 13576d3b2a804c9e9fc3c94169754daf P 5ee2fe427ce344ec92a057fc68336b35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f394707d72a74728bb8540687fd8fa28" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 37723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 01:00:59.127560 28312 consensus_queue.cc:1035] T 13576d3b2a804c9e9fc3c94169754daf P 5ee2fe427ce344ec92a057fc68336b35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aada506276a7447ea035a996041bb122" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 36855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 01:00:59.139195 28318 mvcc.cc:205] Tried to move back new op lower bound from 7332706955771625472 to 7332706955480899584. Current Snapshot: MvccSnapshot[applied={T|T < 7332706955771625472}]
I20260924 01:00:59.142678 28320 mvcc.cc:205] Tried to move back new op lower bound from 7332706955771625472 to 7332706955480899584. Current Snapshot: MvccSnapshot[applied={T|T < 7332706955771625472}]
I20260924 01:00:59.155375 27828 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 01:00:59.155704 27828 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 01:00:59.159559 27828 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0b2d4d8c2f9a400998ef07b4860b3bb8: Sending DeleteTablet for 3 replicas of tablet 13576d3b2a804c9e9fc3c94169754daf
I20260924 01:00:59.160775 27953 tablet_service.cc:1514] Processing DeleteTablet for tablet 13576d3b2a804c9e9fc3c94169754daf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 01:00:59 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:49180
I20260924 01:00:59.160844 28087 tablet_service.cc:1514] Processing DeleteTablet for tablet 13576d3b2a804c9e9fc3c94169754daf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 01:00:59 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:45434
I20260924 01:00:59.161000 28221 tablet_service.cc:1514] Processing DeleteTablet for tablet 13576d3b2a804c9e9fc3c94169754daf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 01:00:59 UTC) from {username='kudu', principal='kudu/127.8.64.190@KRBTEST.COM'} at 127.0.0.1:54038
I20260924 01:00:59.163208 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 27890
I20260924 01:00:59.174243 28327 tablet_replica.cc:330] T 13576d3b2a804c9e9fc3c94169754daf P aada506276a7447ea035a996041bb122: stopping tablet replica
I20260924 01:00:59.175124 28327 raft_consensus.cc:2229] T 13576d3b2a804c9e9fc3c94169754daf P aada506276a7447ea035a996041bb122 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 01:00:59.175937 28327 raft_consensus.cc:2258] T 13576d3b2a804c9e9fc3c94169754daf P aada506276a7447ea035a996041bb122 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 01:00:59.179050 28327 ts_tablet_manager.cc:1905] T 13576d3b2a804c9e9fc3c94169754daf P aada506276a7447ea035a996041bb122: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 01:00:59.188881 28327 ts_tablet_manager.cc:1918] T 13576d3b2a804c9e9fc3c94169754daf P aada506276a7447ea035a996041bb122: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 01:00:59.189226 28327 log.cc:1196] T 13576d3b2a804c9e9fc3c94169754daf P aada506276a7447ea035a996041bb122: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-0/wal/wals/13576d3b2a804c9e9fc3c94169754daf
I20260924 01:00:59.189975 28327 ts_tablet_manager.cc:1939] T 13576d3b2a804c9e9fc3c94169754daf P aada506276a7447ea035a996041bb122: Deleting consensus metadata
I20260924 01:00:59.192425 27813 catalog_manager.cc:4841] TS aada506276a7447ea035a996041bb122 (127.8.64.129:36855): tablet 13576d3b2a804c9e9fc3c94169754daf (table test-table [id=6cca1d65a2a6425d897f56174d2bd1f8]) successfully deleted
Sep 24 01:00:59 dist-test-slave-mmhs krb5kdc[27784](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.129@KRBTEST.COM
I20260924 01:00:59.207309 28328 tablet_replica.cc:330] T 13576d3b2a804c9e9fc3c94169754daf P f394707d72a74728bb8540687fd8fa28: stopping tablet replica
I20260924 01:00:59.208474 28328 raft_consensus.cc:2229] T 13576d3b2a804c9e9fc3c94169754daf P f394707d72a74728bb8540687fd8fa28 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 01:00:59.208600 28329 tablet_replica.cc:330] T 13576d3b2a804c9e9fc3c94169754daf P 5ee2fe427ce344ec92a057fc68336b35: stopping tablet replica
I20260924 01:00:59.209002 28329 raft_consensus.cc:2229] T 13576d3b2a804c9e9fc3c94169754daf P 5ee2fe427ce344ec92a057fc68336b35 [term 1 LEADER]: Raft consensus shutting down.
I20260924 01:00:59.209193 28328 raft_consensus.cc:2258] T 13576d3b2a804c9e9fc3c94169754daf P f394707d72a74728bb8540687fd8fa28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 01:00:59.210286 28329 raft_consensus.cc:2258] T 13576d3b2a804c9e9fc3c94169754daf P 5ee2fe427ce344ec92a057fc68336b35 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 01:00:59.212750 28329 ts_tablet_manager.cc:1905] T 13576d3b2a804c9e9fc3c94169754daf P 5ee2fe427ce344ec92a057fc68336b35: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 01:00:59.213228 28328 ts_tablet_manager.cc:1905] T 13576d3b2a804c9e9fc3c94169754daf P f394707d72a74728bb8540687fd8fa28: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 01:00:59.213562 28300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:59.164582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49190 (local address 127.8.64.129:36855)
0924 01:00:59.164834 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:59.164843 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:59.185000 (+ 20157us) server_negotiation.cc:407] Connection header received
0924 01:00:59.185523 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:59.185530 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:59.185614 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:59.185697 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:59.187503 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:59.194875 (+ 7372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:59.199058 (+ 4183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:59.199715 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:59.199815 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:59.205629 (+ 5814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:59.205647 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:59.205667 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:00:59.205712 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:59.209827 (+ 4115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:59.210793 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:59.210805 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:59.210811 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:59.210980 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:59.211394 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:59.211399 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:59.211402 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:59.211874 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:59.212177 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:59.212954 (+ 777us) server_negotiation.cc:299] Negotiation successful
0924 01:00:59.213174 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260924 01:00:59.214201 28013 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 01:00:59.219980 28328 ts_tablet_manager.cc:1918] T 13576d3b2a804c9e9fc3c94169754daf P f394707d72a74728bb8540687fd8fa28: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 01:00:59.220157 28328 log.cc:1196] T 13576d3b2a804c9e9fc3c94169754daf P f394707d72a74728bb8540687fd8fa28: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-1/wal/wals/13576d3b2a804c9e9fc3c94169754daf
I20260924 01:00:59.220610 28328 ts_tablet_manager.cc:1939] T 13576d3b2a804c9e9fc3c94169754daf P f394707d72a74728bb8540687fd8fa28: Deleting consensus metadata
I20260924 01:00:59.222234 27815 catalog_manager.cc:4841] TS f394707d72a74728bb8540687fd8fa28 (127.8.64.130:37723): tablet 13576d3b2a804c9e9fc3c94169754daf (table test-table [id=6cca1d65a2a6425d897f56174d2bd1f8]) successfully deleted
I20260924 01:00:59.227013 28329 ts_tablet_manager.cc:1918] T 13576d3b2a804c9e9fc3c94169754daf P 5ee2fe427ce344ec92a057fc68336b35: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 01:00:59.227217 28329 log.cc:1196] T 13576d3b2a804c9e9fc3c94169754daf P 5ee2fe427ce344ec92a057fc68336b35: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790211343099518-8450-0/minicluster-data/ts-2/wal/wals/13576d3b2a804c9e9fc3c94169754daf
I20260924 01:00:59.227639 28329 ts_tablet_manager.cc:1939] T 13576d3b2a804c9e9fc3c94169754daf P 5ee2fe427ce344ec92a057fc68336b35: Deleting consensus metadata
I20260924 01:00:59.229156 27814 catalog_manager.cc:4841] TS 5ee2fe427ce344ec92a057fc68336b35 (127.8.64.131:33049): tablet 13576d3b2a804c9e9fc3c94169754daf (table test-table [id=6cca1d65a2a6425d897f56174d2bd1f8]) successfully deleted
I20260924 01:00:59.332787 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 27890
I20260924 01:00:59.353713 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 28024
Sep 24 01:00:59 dist-test-slave-mmhs krb5kdc[27784](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.130@KRBTEST.COM
I20260924 01:00:59.372000 28303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:59.354684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45436 (local address 127.8.64.130:37723)
0924 01:00:59.354887 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:59.354894 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:59.354917 (+ 23us) server_negotiation.cc:407] Connection header received
0924 01:00:59.355071 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:59.355079 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:59.355206 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:59.355327 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:59.357069 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:59.357987 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:59.360009 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:59.360441 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:59.360555 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:59.365765 (+ 5210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:59.365781 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:59.365797 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:00:59.365841 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:59.368944 (+ 3103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:59.369789 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:59.369802 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:59.369806 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:59.369954 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:59.370363 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:59.370369 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:59.370372 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:59.370771 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:59.371089 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:59.371424 (+ 335us) server_negotiation.cc:299] Negotiation successful
0924 01:00:59.371650 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260924 01:00:59.372701 28147 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 01:00:59.486501 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 28024
I20260924 01:00:59.511045 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 28158
Sep 24 01:00:59 dist-test-slave-mmhs krb5kdc[27784](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.131@KRBTEST.COM
I20260924 01:00:59.529282 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:00:59.511817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54050 (local address 127.8.64.131:33049)
0924 01:00:59.511998 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0924 01:00:59.512003 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 01:00:59.512021 (+ 18us) server_negotiation.cc:407] Connection header received
0924 01:00:59.512228 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:00:59.512232 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:00:59.512306 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:00:59.512387 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:00:59.514176 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:59.515129 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:59.517016 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:00:59.517493 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:00:59.517607 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:00:59.523073 (+ 5466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:00:59.523088 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:00:59.523104 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:00:59.523155 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:00:59.526293 (+ 3138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:59.527071 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:59.527085 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:59.527090 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:59.527266 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:00:59.527673 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:00:59.527679 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:00:59.527682 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:00:59.528050 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:00:59.528373 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0924 01:00:59.528673 (+ 300us) server_negotiation.cc:299] Negotiation successful
0924 01:00:59.528900 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260924 01:00:59.529949 28281 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 01:00:59.654047 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 28158
I20260924 01:00:59.671067 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 27797
I20260924 01:00:59.672214 27859 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 01:00:59.781977 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 27797
2026-09-24T01:00:59Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4529 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-24T01:00:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T01:00:59Z Disabled control of system clock
I20260924 01:00:59.862527 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:34483
--webserver_interface=127.8.64.190
--webserver_port=0
--builtin_ntp_servers=127.8.64.148:43793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:34483
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 01:01:00.195230 28338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:01:00.195545 28338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:01:00.195724 28338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:01:00.203195 28338 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 01:01:00.203315 28338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:01:00.203425 28338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:01:00.203532 28338 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 01:01:00.203604 28338 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 01:01:00.213058 28338 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:43793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:34483
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:34483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:01:00.214630 28338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:01:00.216598 28338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:01:00.228957 28343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:00.229022 28344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:00.228991 28346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:01:00.229884 28338 server_base.cc:1034] running on GCE node
I20260924 01:01:00.230734 28338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:01:00.232107 28338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:01:00.233352 28338 hybrid_clock.cc:648] HybridClock initialized: now 1790211660233310 us; error 47 us; skew 500 ppm
I20260924 01:01:00.233731 28338 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 01:01:00.236150 28338 webserver.cc:460] Webserver started at http://127.8.64.190:33929/ using document root <none> and password file <none>
I20260924 01:01:00.236835 28338 fs_manager.cc:359] Metadata directory not provided
I20260924 01:01:00.236970 28338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:01:00.237293 28338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:01:00.240191 28338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "004909555a8f4b8ebebbdf4c30dbe5d2"
format_stamp: "Formatted at 2026-09-24 01:01:00 on dist-test-slave-mmhs"
server_key: "79548ecd49c3ba515e99dcbd5879cffb"
server_key_iv: "6571735056246cf2abcfaead6f77c894"
server_key_version: "encryptionkey@0"
I20260924 01:01:00.240998 28338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "004909555a8f4b8ebebbdf4c30dbe5d2"
format_stamp: "Formatted at 2026-09-24 01:01:00 on dist-test-slave-mmhs"
server_key: "79548ecd49c3ba515e99dcbd5879cffb"
server_key_iv: "6571735056246cf2abcfaead6f77c894"
server_key_version: "encryptionkey@0"
I20260924 01:01:00.246045 28338 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260924 01:01:00.249490 28352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:01:00.250861 28338 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 01:01:00.251060 28338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "004909555a8f4b8ebebbdf4c30dbe5d2"
format_stamp: "Formatted at 2026-09-24 01:01:00 on dist-test-slave-mmhs"
server_key: "79548ecd49c3ba515e99dcbd5879cffb"
server_key_iv: "6571735056246cf2abcfaead6f77c894"
server_key_version: "encryptionkey@0"
I20260924 01:01:00.251271 28338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:01:00.275071 28338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:01:00.275929 28338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:01:00.276247 28338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:01:00.294850 28338 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.190:34483
I20260924 01:01:00.294875 28403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.190:34483 every 8 connection(s)
I20260924 01:01:00.296217 28338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 01:01:00.299858 28404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:01:00.306106 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 28338
I20260924 01:01:00.306315 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 01:01:00.306900 8450 external_mini_cluster.cc:1428] Setting key 537ea4e763e9907b74b3f6977253e5d1
I20260924 01:01:00.310840 28404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 004909555a8f4b8ebebbdf4c30dbe5d2: Bootstrap starting.
I20260924 01:01:00.314102 28404 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 004909555a8f4b8ebebbdf4c30dbe5d2: Neither blocks nor log segments found. Creating new log.
I20260924 01:01:00.315213 28404 log.cc:824] T 00000000000000000000000000000000 P 004909555a8f4b8ebebbdf4c30dbe5d2: Log is configured to *not* fsync() on all Append() calls
I20260924 01:01:00.318219 28404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 004909555a8f4b8ebebbdf4c30dbe5d2: No bootstrap required, opened a new log
I20260924 01:01:00.321309 28407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:00.308957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44652 (local address 127.8.64.190:34483)
0924 01:01:00.309986 (+ 1029us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:00.309997 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:00.310040 (+ 43us) server_negotiation.cc:407] Connection header received
0924 01:01:00.312007 (+ 1967us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:00.312043 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:00.312540 (+ 497us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:00.312880 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:00.313471 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:00.315031 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:00.316632 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:00.317378 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:00.317537 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:00.318837 (+ 1300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:00.318845 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:00.318867 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:01:00.319407 (+ 540us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:00.319490 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:01:00.319512 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:00.319602 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:00.319874 (+ 272us) server_negotiation.cc:299] Negotiation successful
0924 01:01:00.320241 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":385,"threads_started":1}
I20260924 01:01:00.326043 28404 raft_consensus.cc:348] T 00000000000000000000000000000000 P 004909555a8f4b8ebebbdf4c30dbe5d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "004909555a8f4b8ebebbdf4c30dbe5d2" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 34483 } }
I20260924 01:01:00.326442 28404 raft_consensus.cc:374] T 00000000000000000000000000000000 P 004909555a8f4b8ebebbdf4c30dbe5d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 01:01:00.326555 28404 raft_consensus.cc:729] T 00000000000000000000000000000000 P 004909555a8f4b8ebebbdf4c30dbe5d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 004909555a8f4b8ebebbdf4c30dbe5d2, State: Initialized, Role: FOLLOWER
I20260924 01:01:00.327116 28404 consensus_queue.cc:260] T 00000000000000000000000000000000 P 004909555a8f4b8ebebbdf4c30dbe5d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "004909555a8f4b8ebebbdf4c30dbe5d2" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 34483 } }
I20260924 01:01:00.327323 28404 raft_consensus.cc:388] T 00000000000000000000000000000000 P 004909555a8f4b8ebebbdf4c30dbe5d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 01:01:00.327502 28404 raft_consensus.cc:482] T 00000000000000000000000000000000 P 004909555a8f4b8ebebbdf4c30dbe5d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 01:01:00.327687 28404 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 004909555a8f4b8ebebbdf4c30dbe5d2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:01:00.330155 28404 raft_consensus.cc:504] T 00000000000000000000000000000000 P 004909555a8f4b8ebebbdf4c30dbe5d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "004909555a8f4b8ebebbdf4c30dbe5d2" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 34483 } }
I20260924 01:01:00.330699 28404 leader_election.cc:302] T 00000000000000000000000000000000 P 004909555a8f4b8ebebbdf4c30dbe5d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 004909555a8f4b8ebebbdf4c30dbe5d2; no voters:
I20260924 01:01:00.331295 28404 leader_election.cc:288] T 00000000000000000000000000000000 P 004909555a8f4b8ebebbdf4c30dbe5d2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 01:01:00.331478 28409 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 004909555a8f4b8ebebbdf4c30dbe5d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 01:01:00.332211 28409 raft_consensus.cc:686] T 00000000000000000000000000000000 P 004909555a8f4b8ebebbdf4c30dbe5d2 [term 1 LEADER]: Becoming Leader. State: Replica: 004909555a8f4b8ebebbdf4c30dbe5d2, State: Running, Role: LEADER
I20260924 01:01:00.332711 28409 consensus_queue.cc:237] T 00000000000000000000000000000000 P 004909555a8f4b8ebebbdf4c30dbe5d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "004909555a8f4b8ebebbdf4c30dbe5d2" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 34483 } }
I20260924 01:01:00.333333 28404 sys_catalog.cc:564] T 00000000000000000000000000000000 P 004909555a8f4b8ebebbdf4c30dbe5d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 01:01:00.335592 28410 sys_catalog.cc:455] T 00000000000000000000000000000000 P 004909555a8f4b8ebebbdf4c30dbe5d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "004909555a8f4b8ebebbdf4c30dbe5d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "004909555a8f4b8ebebbdf4c30dbe5d2" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 34483 } } }
I20260924 01:01:00.336071 28410 sys_catalog.cc:458] T 00000000000000000000000000000000 P 004909555a8f4b8ebebbdf4c30dbe5d2 [sys.catalog]: This master's current role is: LEADER
I20260924 01:01:00.335917 28411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 004909555a8f4b8ebebbdf4c30dbe5d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 004909555a8f4b8ebebbdf4c30dbe5d2. Latest consensus state: current_term: 1 leader_uuid: "004909555a8f4b8ebebbdf4c30dbe5d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "004909555a8f4b8ebebbdf4c30dbe5d2" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 34483 } } }
I20260924 01:01:00.336242 28411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 004909555a8f4b8ebebbdf4c30dbe5d2 [sys.catalog]: This master's current role is: LEADER
I20260924 01:01:00.338044 28414 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 01:01:00.343551 28414 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 01:01:00.352741 28414 catalog_manager.cc:1347] Generated new cluster ID: 34903a03450d431f964547d67d9c96e4
I20260924 01:01:00.352872 28414 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 01:01:00.362787 28414 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 01:01:00.363999 28414 catalog_manager.cc:1504] Loading token signing keys...
I20260924 01:01:00.383924 28414 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 004909555a8f4b8ebebbdf4c30dbe5d2: Generated new TSK 0
I20260924 01:01:00.384729 28414 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 01:01:00.397845 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.129:0
--local_ip_for_outbound_sockets=127.8.64.129
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:34483
--builtin_ntp_servers=127.8.64.148:43793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 01:01:00.730275 28428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:01:00.730571 28428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:01:00.730736 28428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:01:00.738366 28428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:01:00.738509 28428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:01:00.738735 28428 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.129
I20260924 01:01:00.747833 28428 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:43793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.64.129
--webserver_port=0
--tserver_master_addrs=127.8.64.190:34483
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.129
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:01:00.749240 28428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:01:00.751075 28428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:01:00.764561 28436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:00.764564 28434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:00.764734 28433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:01:00.765281 28428 server_base.cc:1034] running on GCE node
I20260924 01:01:00.765995 28428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:01:00.767078 28428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:01:00.768369 28428 hybrid_clock.cc:648] HybridClock initialized: now 1790211660768276 us; error 49 us; skew 500 ppm
I20260924 01:01:00.768728 28428 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 01:01:00.771020 28428 webserver.cc:460] Webserver started at http://127.8.64.129:45977/ using document root <none> and password file <none>
I20260924 01:01:00.771721 28428 fs_manager.cc:359] Metadata directory not provided
I20260924 01:01:00.771857 28428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:01:00.772182 28428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:01:00.775128 28428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-0/data/instance:
uuid: "155e52068f764292995bd7c462547359"
format_stamp: "Formatted at 2026-09-24 01:01:00 on dist-test-slave-mmhs"
server_key: "d9aee177335f66ea799e64bf1d1096f9"
server_key_iv: "f939ae072ab778bc1986274bec2ee5c8"
server_key_version: "encryptionkey@0"
I20260924 01:01:00.775944 28428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance:
uuid: "155e52068f764292995bd7c462547359"
format_stamp: "Formatted at 2026-09-24 01:01:00 on dist-test-slave-mmhs"
server_key: "d9aee177335f66ea799e64bf1d1096f9"
server_key_iv: "f939ae072ab778bc1986274bec2ee5c8"
server_key_version: "encryptionkey@0"
I20260924 01:01:00.781293 28428 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260924 01:01:00.785408 28442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:01:00.786850 28428 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260924 01:01:00.787034 28428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-0/wal
uuid: "155e52068f764292995bd7c462547359"
format_stamp: "Formatted at 2026-09-24 01:01:00 on dist-test-slave-mmhs"
server_key: "d9aee177335f66ea799e64bf1d1096f9"
server_key_iv: "f939ae072ab778bc1986274bec2ee5c8"
server_key_version: "encryptionkey@0"
I20260924 01:01:00.787281 28428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:01:00.799995 28428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:01:00.800835 28428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:01:00.801151 28428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:01:00.802110 28428 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 01:01:00.803951 28428 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 01:01:00.804068 28428 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 01:01:00.804186 28428 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 01:01:00.804268 28428 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 01:01:00.839129 28428 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.129:34249
I20260924 01:01:00.839169 28554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.129:34249 every 8 connection(s)
I20260924 01:01:00.840541 28428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-0/data/info.pb
I20260924 01:01:00.844096 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 28428
I20260924 01:01:00.844269 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-0/wal/instance
I20260924 01:01:00.844843 8450 external_mini_cluster.cc:1428] Setting key f384cb5d19754cc053b44e95373abcd3
I20260924 01:01:00.847548 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.130:0
--local_ip_for_outbound_sockets=127.8.64.130
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:34483
--builtin_ntp_servers=127.8.64.148:43793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260924 01:01:00.855192 28557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:00.843522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.129:35507 (local address 127.8.64.190:34483)
0924 01:01:00.844059 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:00.844066 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:00.845657 (+ 1591us) server_negotiation.cc:407] Connection header received
0924 01:01:00.847687 (+ 2030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:00.847696 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:00.847790 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:00.847904 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:00.849013 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:00.850194 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:00.851898 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:00.852640 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:00.852784 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:00.854222 (+ 1438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:00.854232 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:00.854239 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:01:00.854266 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:00.854347 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:01:00.854366 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:00.854461 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:00.854707 (+ 246us) server_negotiation.cc:299] Negotiation successful
0924 01:01:00.854815 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":267,"threads_started":1}
I20260924 01:01:00.856266 28558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:00.844242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:34483 (local address 127.8.64.129:35507)
0924 01:01:00.845479 (+ 1237us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:00.845534 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0924 01:01:00.847402 (+ 1868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:01:00.848132 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:01:00.848141 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:01:00.848235 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0924 01:01:00.848831 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:00.848849 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:00.850401 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:00.850405 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:00.851743 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:00.851755 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:00.852852 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:00.852857 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:00.853080 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:00.853880 (+ 800us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 01:01:00.853882 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0924 01:01:00.853924 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 01:01:00.854021 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 01:01:00.854455 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 01:01:00.854464 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 01:01:00.854471 (+ 7us) client_negotiation.cc:769] Sending connection context
0924 01:01:00.854691 (+ 220us) client_negotiation.cc:240] Negotiation successful
0924 01:01:00.855138 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"thread_start_us":403,"threads_started":1}
I20260924 01:01:00.858040 28555 heartbeater.cc:347] Connected to a master server at 127.8.64.190:34483
I20260924 01:01:00.858407 28555 heartbeater.cc:464] Registering TS with master...
I20260924 01:01:00.859318 28555 heartbeater.cc:511] Master 127.8.64.190:34483 requested a full tablet report, sending...
I20260924 01:01:00.861552 28369 ts_manager.cc:194] Registered new tserver with Master: 155e52068f764292995bd7c462547359 (127.8.64.129:34249)
I20260924 01:01:00.863267 28369 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.64.129:35507
W20260924 01:01:01.180390 28559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:01:01.180678 28559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:01:01.180825 28559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:01:01.188609 28559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:01:01.188746 28559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:01:01.188947 28559 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.130
I20260924 01:01:01.200181 28559 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:43793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.64.130
--webserver_port=0
--tserver_master_addrs=127.8.64.190:34483
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.130
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:01:01.201459 28559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:01:01.203239 28559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:01:01.216347 28567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:01.216311 28565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:01.216686 28564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:01:01.217058 28559 server_base.cc:1034] running on GCE node
I20260924 01:01:01.217644 28559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:01:01.218645 28559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:01:01.219887 28559 hybrid_clock.cc:648] HybridClock initialized: now 1790211661219840 us; error 35 us; skew 500 ppm
I20260924 01:01:01.220212 28559 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 01:01:01.222501 28559 webserver.cc:460] Webserver started at http://127.8.64.130:42539/ using document root <none> and password file <none>
I20260924 01:01:01.223179 28559 fs_manager.cc:359] Metadata directory not provided
I20260924 01:01:01.223307 28559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:01:01.223627 28559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:01:01.226727 28559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-1/data/instance:
uuid: "c456b4fca4f64ee6b7c06aa6abc06afc"
format_stamp: "Formatted at 2026-09-24 01:01:01 on dist-test-slave-mmhs"
server_key: "900c16402cbbf98310e8156ca2bb2cba"
server_key_iv: "f4beca58a76d2bae73b073ac8f74adbb"
server_key_version: "encryptionkey@0"
I20260924 01:01:01.227522 28559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance:
uuid: "c456b4fca4f64ee6b7c06aa6abc06afc"
format_stamp: "Formatted at 2026-09-24 01:01:01 on dist-test-slave-mmhs"
server_key: "900c16402cbbf98310e8156ca2bb2cba"
server_key_iv: "f4beca58a76d2bae73b073ac8f74adbb"
server_key_version: "encryptionkey@0"
I20260924 01:01:01.232510 28559 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260924 01:01:01.235970 28573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:01:01.237270 28559 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260924 01:01:01.237542 28559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-1/wal
uuid: "c456b4fca4f64ee6b7c06aa6abc06afc"
format_stamp: "Formatted at 2026-09-24 01:01:01 on dist-test-slave-mmhs"
server_key: "900c16402cbbf98310e8156ca2bb2cba"
server_key_iv: "f4beca58a76d2bae73b073ac8f74adbb"
server_key_version: "encryptionkey@0"
I20260924 01:01:01.237757 28559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:01:01.260643 28559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:01:01.261458 28559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:01:01.261780 28559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:01:01.262769 28559 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 01:01:01.264173 28559 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 01:01:01.264294 28559 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 01:01:01.264432 28559 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 01:01:01.264528 28559 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 01:01:01.298642 28559 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.130:34059
I20260924 01:01:01.298705 28685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.130:34059 every 8 connection(s)
I20260924 01:01:01.299896 28559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-1/data/info.pb
I20260924 01:01:01.307350 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 28559
I20260924 01:01:01.307557 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-1/wal/instance
I20260924 01:01:01.308005 8450 external_mini_cluster.cc:1428] Setting key ba263c6a0691d3a93ac23f4688910690
I20260924 01:01:01.311131 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.64.131:0
--local_ip_for_outbound_sockets=127.8.64.131
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:34483
--builtin_ntp_servers=127.8.64.148:43793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260924 01:01:01.314546 28557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:01.302473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.130:58881 (local address 127.8.64.190:34483)
0924 01:01:01.302685 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:01.302692 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:01.304245 (+ 1553us) server_negotiation.cc:407] Connection header received
0924 01:01:01.306771 (+ 2526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:01.306775 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:01.306838 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:01.306922 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:01.308253 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:01.309219 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:01.311374 (+ 2155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:01.312106 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:01.312256 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:01.313651 (+ 1395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:01.313660 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:01.313667 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:01:01.313692 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:01.313731 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:01:01.313748 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:01.313840 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:01.314031 (+ 191us) server_negotiation.cc:299] Negotiation successful
0924 01:01:01.314141 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260924 01:01:01.315382 28688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:01.303041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:34483 (local address 127.8.64.130:58881)
0924 01:01:01.304065 (+ 1024us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:01.304114 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0924 01:01:01.306499 (+ 2385us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:01:01.307190 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:01:01.307201 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:01:01.307321 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0924 01:01:01.308014 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:01.308038 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:01.309373 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:01.309378 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:01.311195 (+ 1817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:01.311214 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:01.312349 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:01.312384 (+ 35us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:01.312585 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:01.313329 (+ 744us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 01:01:01.313331 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0924 01:01:01.313383 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 01:01:01.313461 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 01:01:01.313842 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 01:01:01.313849 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 01:01:01.313855 (+ 6us) client_negotiation.cc:769] Sending connection context
0924 01:01:01.313992 (+ 137us) client_negotiation.cc:240] Negotiation successful
0924 01:01:01.314262 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":301,"threads_started":1}
I20260924 01:01:01.316748 28686 heartbeater.cc:347] Connected to a master server at 127.8.64.190:34483
I20260924 01:01:01.317050 28686 heartbeater.cc:464] Registering TS with master...
I20260924 01:01:01.317747 28686 heartbeater.cc:511] Master 127.8.64.190:34483 requested a full tablet report, sending...
I20260924 01:01:01.319114 28369 ts_manager.cc:194] Registered new tserver with Master: c456b4fca4f64ee6b7c06aa6abc06afc (127.8.64.130:34059)
I20260924 01:01:01.319921 28369 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.64.130:58881
W20260924 01:01:01.639387 28689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:01:01.639708 28689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:01:01.639899 28689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:01:01.647133 28689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:01:01.647272 28689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:01:01.647456 28689 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.64.131
I20260924 01:01:01.656831 28689 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:43793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.64.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.64.131
--webserver_port=0
--tserver_master_addrs=127.8.64.190:34483
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.64.131
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:01:01.658202 28689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:01:01.660005 28689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:01:01.673523 28694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:01.673574 28695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:01.673465 28697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:01:01.674762 28689 server_base.cc:1034] running on GCE node
I20260924 01:01:01.675329 28689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:01:01.676241 28689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:01:01.677443 28689 hybrid_clock.cc:648] HybridClock initialized: now 1790211661677390 us; error 53 us; skew 500 ppm
I20260924 01:01:01.677716 28689 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260924 01:01:01.679872 28689 webserver.cc:460] Webserver started at http://127.8.64.131:34779/ using document root <none> and password file <none>
I20260924 01:01:01.680500 28689 fs_manager.cc:359] Metadata directory not provided
I20260924 01:01:01.680588 28689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:01:01.680841 28689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:01:01.683699 28689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-2/data/instance:
uuid: "bf2354afa699432899105131c2d64759"
format_stamp: "Formatted at 2026-09-24 01:01:01 on dist-test-slave-mmhs"
server_key: "43d83b5546c8dd343895af1acde899d4"
server_key_iv: "26661f465c69a22515c6081414952179"
server_key_version: "encryptionkey@0"
I20260924 01:01:01.684419 28689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance:
uuid: "bf2354afa699432899105131c2d64759"
format_stamp: "Formatted at 2026-09-24 01:01:01 on dist-test-slave-mmhs"
server_key: "43d83b5546c8dd343895af1acde899d4"
server_key_iv: "26661f465c69a22515c6081414952179"
server_key_version: "encryptionkey@0"
I20260924 01:01:01.689471 28689 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260924 01:01:01.693040 28703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:01:01.694434 28689 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 01:01:01.694598 28689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-2/wal
uuid: "bf2354afa699432899105131c2d64759"
format_stamp: "Formatted at 2026-09-24 01:01:01 on dist-test-slave-mmhs"
server_key: "43d83b5546c8dd343895af1acde899d4"
server_key_iv: "26661f465c69a22515c6081414952179"
server_key_version: "encryptionkey@0"
I20260924 01:01:01.694818 28689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:01:01.711429 28689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:01:01.712213 28689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:01:01.712524 28689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:01:01.713480 28689 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 01:01:01.714929 28689 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 01:01:01.715051 28689 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 01:01:01.715176 28689 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 01:01:01.715269 28689 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 01:01:01.748411 28689 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.131:41519
I20260924 01:01:01.748446 28815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.131:41519 every 8 connection(s)
I20260924 01:01:01.749799 28689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-2/data/info.pb
I20260924 01:01:01.758206 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 28689
I20260924 01:01:01.758441 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-2/wal/instance
I20260924 01:01:01.758874 8450 external_mini_cluster.cc:1428] Setting key 69f2117f6ce2f71e12bf8530e7c2b3fe
I20260924 01:01:01.763491 28557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:01.752567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:41751 (local address 127.8.64.190:34483)
0924 01:01:01.752757 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:01.752763 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:01.754335 (+ 1572us) server_negotiation.cc:407] Connection header received
0924 01:01:01.756265 (+ 1930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:01.756271 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:01.756370 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:01.756485 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:01.757570 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:01.758448 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:01.760492 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:01.761048 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:01.761145 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:01.762680 (+ 1535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:01.762685 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:01.762688 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:01:01.762706 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:01.762738 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:01:01.762750 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:01.762823 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:01.763138 (+ 315us) server_negotiation.cc:299] Negotiation successful
0924 01:01:01.763221 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260924 01:01:01.764480 28818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:01.753159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:34483 (local address 127.8.64.131:41751)
0924 01:01:01.754111 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:01.754149 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0924 01:01:01.755966 (+ 1817us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:01:01.756732 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:01:01.756740 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:01:01.756832 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0924 01:01:01.757403 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:01.757419 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:01.758655 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:01.758662 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:01.760289 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:01.760306 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:01.761287 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:01.761295 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:01.761600 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:01.762357 (+ 757us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 01:01:01.762361 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 01:01:01.762420 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 01:01:01.762503 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 01:01:01.762864 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 01:01:01.762873 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 01:01:01.762879 (+ 6us) client_negotiation.cc:769] Sending connection context
0924 01:01:01.763054 (+ 175us) client_negotiation.cc:240] Negotiation successful
0924 01:01:01.763389 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":330,"threads_started":1}
I20260924 01:01:01.765908 28816 heartbeater.cc:347] Connected to a master server at 127.8.64.190:34483
I20260924 01:01:01.766223 28816 heartbeater.cc:464] Registering TS with master...
I20260924 01:01:01.767053 28816 heartbeater.cc:511] Master 127.8.64.190:34483 requested a full tablet report, sending...
I20260924 01:01:01.768558 28369 ts_manager.cc:194] Registered new tserver with Master: bf2354afa699432899105131c2d64759 (127.8.64.131:41519)
I20260924 01:01:01.769447 28369 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.64.131:41751
I20260924 01:01:01.774528 8450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 01:01:01.784421 28557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:01.778772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44662 (local address 127.8.64.190:34483)
0924 01:01:01.779019 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:01.779026 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:01.779310 (+ 284us) server_negotiation.cc:407] Connection header received
0924 01:01:01.779482 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:01.779488 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:01.779565 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:01.779646 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:01.780089 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:01.780761 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:01.782109 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:01.782549 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:01.782687 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:01.783742 (+ 1055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:01.783747 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:01.783751 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:01:01.783768 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:01.783799 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:01:01.783811 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:01.783903 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:01.784044 (+ 141us) server_negotiation.cc:299] Negotiation successful
0924 01:01:01.784134 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260924 01:01:01.788300 28369 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44662:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 01:01:01.795296 28369 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 01:01:01.818205 28829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:01.807600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58110 (local address 127.8.64.131:41519)
0924 01:01:01.808888 (+ 1288us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:01.808894 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:01.808948 (+ 54us) server_negotiation.cc:407] Connection header received
0924 01:01:01.809061 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:01.809073 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:01.809345 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:01.809549 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 01:01:01.810240 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:01.811821 (+ 1581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:01.814755 (+ 2934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:01.816271 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:01.816388 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:01.816671 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:01.817615 (+ 944us) server_negotiation.cc:299] Negotiation successful
0924 01:01:01.817784 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1135,"thread_start_us":993,"threads_started":1}
I20260924 01:01:01.819315 28828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:01.807321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.131:41519 (local address 127.0.0.1:58110)
0924 01:01:01.808401 (+ 1080us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:01.808436 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 01:01:01.808612 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:01:01.809635 (+ 1023us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:01:01.809642 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:01:01.809682 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 01:01:01.810030 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:01.810044 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:01.811998 (+ 1954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:01.812003 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:01.814546 (+ 2543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:01.814560 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:01.817124 (+ 2564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:01.817130 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:01.817343 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:01.817392 (+ 49us) client_negotiation.cc:769] Sending connection context
0924 01:01:01.818749 (+ 1357us) client_negotiation.cc:240] Negotiation successful
0924 01:01:01.818958 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":849,"thread_start_us":698,"threads_started":1}
I20260924 01:01:01.821841 28830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:01.808799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.130:34059 (local address 127.0.0.1:52680)
0924 01:01:01.809331 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:01.809349 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0924 01:01:01.809473 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:01:01.812195 (+ 2722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:01:01.812199 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:01:01.812231 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 01:01:01.812728 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:01.812747 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:01.815814 (+ 3067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:01.815819 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:01.817939 (+ 2120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:01.817953 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:01.820914 (+ 2961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:01.820920 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:01.821154 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:01.821280 (+ 126us) client_negotiation.cc:769] Sending connection context
0924 01:01:01.821381 (+ 101us) client_negotiation.cc:240] Negotiation successful
0924 01:01:01.821509 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400,"mutex_wait_us":59,"thread_start_us":271,"threads_started":1}
I20260924 01:01:01.821988 28831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:01.809256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52680 (local address 127.8.64.130:34059)
0924 01:01:01.810214 (+ 958us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:01.810229 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:01.810258 (+ 29us) server_negotiation.cc:407] Connection header received
0924 01:01:01.811441 (+ 1183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:01.811450 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:01.811692 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:01.811881 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 01:01:01.812956 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:01.815638 (+ 2682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:01.819227 (+ 3589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:01.820710 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:01.820860 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:01.821127 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:01.821449 (+ 322us) server_negotiation.cc:299] Negotiation successful
0924 01:01:01.821645 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":771,"thread_start_us":628,"threads_started":1}
I20260924 01:01:01.824131 28751 tablet_service.cc:1467] Processing CreateTablet for tablet ae853c1bf9b5480eb8b46979dc982bb9 (DEFAULT_TABLE table=test-table [id=902a0555f58249188778c0732185089c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 01:01:01.825731 28751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae853c1bf9b5480eb8b46979dc982bb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:01:01.825697 28621 tablet_service.cc:1467] Processing CreateTablet for tablet ae853c1bf9b5480eb8b46979dc982bb9 (DEFAULT_TABLE table=test-table [id=902a0555f58249188778c0732185089c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 01:01:01.826975 28828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:01.809370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:34249 (local address 127.0.0.1:45082)
0924 01:01:01.819756 (+ 10386us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:01.819775 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0924 01:01:01.819868 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:01:01.821041 (+ 1173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:01:01.821046 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:01:01.821073 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 01:01:01.821305 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:01.821325 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:01.823253 (+ 1928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:01.823261 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:01.824828 (+ 1567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:01.824865 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:01.826174 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:01.826180 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:01.826391 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:01.826422 (+ 31us) client_negotiation.cc:769] Sending connection context
0924 01:01:01.826509 (+ 87us) client_negotiation.cc:240] Negotiation successful
0924 01:01:01.826651 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10260,"mutex_wait_us":4,"thread_start_us":10392,"threads_started":1}
I20260924 01:01:01.827049 28833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:01.809965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45082 (local address 127.8.64.129:34249)
0924 01:01:01.820382 (+ 10417us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:01.820392 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:01.820422 (+ 30us) server_negotiation.cc:407] Connection header received
0924 01:01:01.820533 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:01.820543 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:01.820808 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:01.821530 (+ 722us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 01:01:01.821776 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:01.823081 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:01.825010 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:01.826008 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:01.826173 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:01.826470 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:01.826561 (+ 91us) server_negotiation.cc:299] Negotiation successful
0924 01:01:01.826716 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10247,"thread_start_us":10120,"threads_started":1}
I20260924 01:01:01.827142 28621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae853c1bf9b5480eb8b46979dc982bb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:01:01.830459 28490 tablet_service.cc:1467] Processing CreateTablet for tablet ae853c1bf9b5480eb8b46979dc982bb9 (DEFAULT_TABLE table=test-table [id=902a0555f58249188778c0732185089c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 01:01:01.831534 28490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae853c1bf9b5480eb8b46979dc982bb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:01:01.847227 28835 tablet_bootstrap.cc:492] T ae853c1bf9b5480eb8b46979dc982bb9 P bf2354afa699432899105131c2d64759: Bootstrap starting.
I20260924 01:01:01.847227 28834 tablet_bootstrap.cc:492] T ae853c1bf9b5480eb8b46979dc982bb9 P c456b4fca4f64ee6b7c06aa6abc06afc: Bootstrap starting.
I20260924 01:01:01.850919 28834 tablet_bootstrap.cc:654] T ae853c1bf9b5480eb8b46979dc982bb9 P c456b4fca4f64ee6b7c06aa6abc06afc: Neither blocks nor log segments found. Creating new log.
I20260924 01:01:01.852629 28835 tablet_bootstrap.cc:654] T ae853c1bf9b5480eb8b46979dc982bb9 P bf2354afa699432899105131c2d64759: Neither blocks nor log segments found. Creating new log.
I20260924 01:01:01.853498 28836 tablet_bootstrap.cc:492] T ae853c1bf9b5480eb8b46979dc982bb9 P 155e52068f764292995bd7c462547359: Bootstrap starting.
I20260924 01:01:01.853564 28834 log.cc:824] T ae853c1bf9b5480eb8b46979dc982bb9 P c456b4fca4f64ee6b7c06aa6abc06afc: Log is configured to *not* fsync() on all Append() calls
I20260924 01:01:01.855330 28835 log.cc:824] T ae853c1bf9b5480eb8b46979dc982bb9 P bf2354afa699432899105131c2d64759: Log is configured to *not* fsync() on all Append() calls
I20260924 01:01:01.856943 28834 tablet_bootstrap.cc:492] T ae853c1bf9b5480eb8b46979dc982bb9 P c456b4fca4f64ee6b7c06aa6abc06afc: No bootstrap required, opened a new log
I20260924 01:01:01.857079 28836 tablet_bootstrap.cc:654] T ae853c1bf9b5480eb8b46979dc982bb9 P 155e52068f764292995bd7c462547359: Neither blocks nor log segments found. Creating new log.
I20260924 01:01:01.857348 28834 ts_tablet_manager.cc:1397] T ae853c1bf9b5480eb8b46979dc982bb9 P c456b4fca4f64ee6b7c06aa6abc06afc: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260924 01:01:01.858328 28836 log.cc:824] T ae853c1bf9b5480eb8b46979dc982bb9 P 155e52068f764292995bd7c462547359: Log is configured to *not* fsync() on all Append() calls
I20260924 01:01:01.860436 28835 tablet_bootstrap.cc:492] T ae853c1bf9b5480eb8b46979dc982bb9 P bf2354afa699432899105131c2d64759: No bootstrap required, opened a new log
I20260924 01:01:01.860714 28835 ts_tablet_manager.cc:1397] T ae853c1bf9b5480eb8b46979dc982bb9 P bf2354afa699432899105131c2d64759: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.009s
I20260924 01:01:01.864878 28836 tablet_bootstrap.cc:492] T ae853c1bf9b5480eb8b46979dc982bb9 P 155e52068f764292995bd7c462547359: No bootstrap required, opened a new log
I20260924 01:01:01.865135 28836 ts_tablet_manager.cc:1397] T ae853c1bf9b5480eb8b46979dc982bb9 P 155e52068f764292995bd7c462547359: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260924 01:01:01.866987 28555 heartbeater.cc:503] Master 127.8.64.190:34483 was elected leader, sending a full tablet report...
I20260924 01:01:01.868638 28834 raft_consensus.cc:348] T ae853c1bf9b5480eb8b46979dc982bb9 P c456b4fca4f64ee6b7c06aa6abc06afc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf2354afa699432899105131c2d64759" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 41519 } } peers { permanent_uuid: "c456b4fca4f64ee6b7c06aa6abc06afc" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 34059 } } peers { permanent_uuid: "155e52068f764292995bd7c462547359" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 34249 } }
I20260924 01:01:01.868685 28835 raft_consensus.cc:348] T ae853c1bf9b5480eb8b46979dc982bb9 P bf2354afa699432899105131c2d64759 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf2354afa699432899105131c2d64759" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 41519 } } peers { permanent_uuid: "c456b4fca4f64ee6b7c06aa6abc06afc" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 34059 } } peers { permanent_uuid: "155e52068f764292995bd7c462547359" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 34249 } }
I20260924 01:01:01.869086 28835 raft_consensus.cc:374] T ae853c1bf9b5480eb8b46979dc982bb9 P bf2354afa699432899105131c2d64759 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 01:01:01.869199 28834 raft_consensus.cc:374] T ae853c1bf9b5480eb8b46979dc982bb9 P c456b4fca4f64ee6b7c06aa6abc06afc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 01:01:01.869249 28835 raft_consensus.cc:729] T ae853c1bf9b5480eb8b46979dc982bb9 P bf2354afa699432899105131c2d64759 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf2354afa699432899105131c2d64759, State: Initialized, Role: FOLLOWER
I20260924 01:01:01.869365 28834 raft_consensus.cc:729] T ae853c1bf9b5480eb8b46979dc982bb9 P c456b4fca4f64ee6b7c06aa6abc06afc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c456b4fca4f64ee6b7c06aa6abc06afc, State: Initialized, Role: FOLLOWER
I20260924 01:01:01.870093 28835 consensus_queue.cc:260] T ae853c1bf9b5480eb8b46979dc982bb9 P bf2354afa699432899105131c2d64759 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf2354afa699432899105131c2d64759" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 41519 } } peers { permanent_uuid: "c456b4fca4f64ee6b7c06aa6abc06afc" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 34059 } } peers { permanent_uuid: "155e52068f764292995bd7c462547359" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 34249 } }
I20260924 01:01:01.870102 28834 consensus_queue.cc:260] T ae853c1bf9b5480eb8b46979dc982bb9 P c456b4fca4f64ee6b7c06aa6abc06afc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf2354afa699432899105131c2d64759" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 41519 } } peers { permanent_uuid: "c456b4fca4f64ee6b7c06aa6abc06afc" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 34059 } } peers { permanent_uuid: "155e52068f764292995bd7c462547359" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 34249 } }
I20260924 01:01:01.873582 28816 heartbeater.cc:503] Master 127.8.64.190:34483 was elected leader, sending a full tablet report...
I20260924 01:01:01.873628 28686 heartbeater.cc:503] Master 127.8.64.190:34483 was elected leader, sending a full tablet report...
I20260924 01:01:01.873836 28834 ts_tablet_manager.cc:1428] T ae853c1bf9b5480eb8b46979dc982bb9 P c456b4fca4f64ee6b7c06aa6abc06afc: Time spent starting tablet: real 0.016s user 0.011s sys 0.004s
I20260924 01:01:01.875240 28835 ts_tablet_manager.cc:1428] T ae853c1bf9b5480eb8b46979dc982bb9 P bf2354afa699432899105131c2d64759: Time spent starting tablet: real 0.014s user 0.004s sys 0.010s
I20260924 01:01:01.877786 28836 raft_consensus.cc:348] T ae853c1bf9b5480eb8b46979dc982bb9 P 155e52068f764292995bd7c462547359 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf2354afa699432899105131c2d64759" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 41519 } } peers { permanent_uuid: "c456b4fca4f64ee6b7c06aa6abc06afc" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 34059 } } peers { permanent_uuid: "155e52068f764292995bd7c462547359" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 34249 } }
I20260924 01:01:01.878280 28836 raft_consensus.cc:374] T ae853c1bf9b5480eb8b46979dc982bb9 P 155e52068f764292995bd7c462547359 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 01:01:01.878420 28836 raft_consensus.cc:729] T ae853c1bf9b5480eb8b46979dc982bb9 P 155e52068f764292995bd7c462547359 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 155e52068f764292995bd7c462547359, State: Initialized, Role: FOLLOWER
I20260924 01:01:01.879034 28836 consensus_queue.cc:260] T ae853c1bf9b5480eb8b46979dc982bb9 P 155e52068f764292995bd7c462547359 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf2354afa699432899105131c2d64759" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 41519 } } peers { permanent_uuid: "c456b4fca4f64ee6b7c06aa6abc06afc" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 34059 } } peers { permanent_uuid: "155e52068f764292995bd7c462547359" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 34249 } }
I20260924 01:01:01.880266 28836 ts_tablet_manager.cc:1428] T ae853c1bf9b5480eb8b46979dc982bb9 P 155e52068f764292995bd7c462547359: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
I20260924 01:01:01.897315 28841 raft_consensus.cc:482] T ae853c1bf9b5480eb8b46979dc982bb9 P bf2354afa699432899105131c2d64759 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 01:01:01.897632 28841 raft_consensus.cc:504] T ae853c1bf9b5480eb8b46979dc982bb9 P bf2354afa699432899105131c2d64759 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf2354afa699432899105131c2d64759" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 41519 } } peers { permanent_uuid: "c456b4fca4f64ee6b7c06aa6abc06afc" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 34059 } } peers { permanent_uuid: "155e52068f764292995bd7c462547359" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 34249 } }
I20260924 01:01:01.899405 28841 leader_election.cc:288] T ae853c1bf9b5480eb8b46979dc982bb9 P bf2354afa699432899105131c2d64759 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c456b4fca4f64ee6b7c06aa6abc06afc (127.8.64.130:34059), 155e52068f764292995bd7c462547359 (127.8.64.129:34249)
I20260924 01:01:01.906126 28831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:01.899933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:34251 (local address 127.8.64.130:34059)
0924 01:01:01.900170 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:01.900177 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:01.900199 (+ 22us) server_negotiation.cc:407] Connection header received
0924 01:01:01.900399 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:01.900404 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:01.900464 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:01.900576 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 01:01:01.901150 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:01.902145 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:01.904085 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:01.905145 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:01.905257 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:01.905304 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:01.905755 (+ 451us) server_negotiation.cc:299] Negotiation successful
0924 01:01:01.905853 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260924 01:01:01.906149 28818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:01.899874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.130:34059 (local address 127.8.64.131:34251)
0924 01:01:01.900066 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:01.900084 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0924 01:01:01.900217 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:01:01.900674 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:01:01.900680 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:01:01.900716 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 01:01:01.900958 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:01.900967 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:01.902280 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:01.902285 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:01.903934 (+ 1649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:01.903945 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:01.905311 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:01.905318 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:01.905523 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:01.905564 (+ 41us) client_negotiation.cc:769] Sending connection context
0924 01:01:01.905679 (+ 115us) client_negotiation.cc:240] Negotiation successful
0924 01:01:01.905824 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260924 01:01:01.906991 28833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:01.900524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.64.131:41333 (local address 127.8.64.129:34249)
0924 01:01:01.900683 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:01.900689 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:01.901031 (+ 342us) server_negotiation.cc:407] Connection header received
0924 01:01:01.901175 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:01.901179 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:01.901245 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:01.901312 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 01:01:01.901921 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:01.902846 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:01.904820 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:01.905851 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:01.906043 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:01.906102 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:01.906630 (+ 528us) server_negotiation.cc:299] Negotiation successful
0924 01:01:01.906751 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260924 01:01:01.907541 28843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:01.900239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.129:34249 (local address 127.8.64.131:41333)
0924 01:01:01.900928 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:01.900941 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 01:01:01.901037 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:01:01.901349 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:01:01.901353 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:01:01.901377 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 01:01:01.901738 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:01.901749 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:01.903021 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:01.903025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:01.904611 (+ 1586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:01.904624 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:01.906211 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:01.906216 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:01.906434 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:01.906475 (+ 41us) client_negotiation.cc:769] Sending connection context
0924 01:01:01.906587 (+ 112us) client_negotiation.cc:240] Negotiation successful
0924 01:01:01.906708 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":438,"threads_started":1}
I20260924 01:01:01.907440 28641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae853c1bf9b5480eb8b46979dc982bb9" candidate_uuid: "bf2354afa699432899105131c2d64759" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c456b4fca4f64ee6b7c06aa6abc06afc" is_pre_election: true
I20260924 01:01:01.908030 28641 raft_consensus.cc:2454] T ae853c1bf9b5480eb8b46979dc982bb9 P c456b4fca4f64ee6b7c06aa6abc06afc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf2354afa699432899105131c2d64759 in term 0.
I20260924 01:01:01.908668 28510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae853c1bf9b5480eb8b46979dc982bb9" candidate_uuid: "bf2354afa699432899105131c2d64759" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "155e52068f764292995bd7c462547359" is_pre_election: true
I20260924 01:01:01.908787 28707 leader_election.cc:302] T ae853c1bf9b5480eb8b46979dc982bb9 P bf2354afa699432899105131c2d64759 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bf2354afa699432899105131c2d64759, c456b4fca4f64ee6b7c06aa6abc06afc; no voters:
I20260924 01:01:01.909145 28510 raft_consensus.cc:2454] T ae853c1bf9b5480eb8b46979dc982bb9 P 155e52068f764292995bd7c462547359 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf2354afa699432899105131c2d64759 in term 0.
I20260924 01:01:01.909209 28841 raft_consensus.cc:2781] T ae853c1bf9b5480eb8b46979dc982bb9 P bf2354afa699432899105131c2d64759 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 01:01:01.909391 28841 raft_consensus.cc:482] T ae853c1bf9b5480eb8b46979dc982bb9 P bf2354afa699432899105131c2d64759 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 01:01:01.909539 28841 raft_consensus.cc:3037] T ae853c1bf9b5480eb8b46979dc982bb9 P bf2354afa699432899105131c2d64759 [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:01:01.912030 28841 raft_consensus.cc:504] T ae853c1bf9b5480eb8b46979dc982bb9 P bf2354afa699432899105131c2d64759 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf2354afa699432899105131c2d64759" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 41519 } } peers { permanent_uuid: "c456b4fca4f64ee6b7c06aa6abc06afc" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 34059 } } peers { permanent_uuid: "155e52068f764292995bd7c462547359" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 34249 } }
I20260924 01:01:01.912680 28841 leader_election.cc:288] T ae853c1bf9b5480eb8b46979dc982bb9 P bf2354afa699432899105131c2d64759 [CANDIDATE]: Term 1 election: Requested vote from peers c456b4fca4f64ee6b7c06aa6abc06afc (127.8.64.130:34059), 155e52068f764292995bd7c462547359 (127.8.64.129:34249)
I20260924 01:01:01.913132 28641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae853c1bf9b5480eb8b46979dc982bb9" candidate_uuid: "bf2354afa699432899105131c2d64759" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c456b4fca4f64ee6b7c06aa6abc06afc"
I20260924 01:01:01.913203 28510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae853c1bf9b5480eb8b46979dc982bb9" candidate_uuid: "bf2354afa699432899105131c2d64759" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "155e52068f764292995bd7c462547359"
I20260924 01:01:01.913338 28641 raft_consensus.cc:3037] T ae853c1bf9b5480eb8b46979dc982bb9 P c456b4fca4f64ee6b7c06aa6abc06afc [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:01:01.913456 28510 raft_consensus.cc:3037] T ae853c1bf9b5480eb8b46979dc982bb9 P 155e52068f764292995bd7c462547359 [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:01:01.915721 28641 raft_consensus.cc:2454] T ae853c1bf9b5480eb8b46979dc982bb9 P c456b4fca4f64ee6b7c06aa6abc06afc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf2354afa699432899105131c2d64759 in term 1.
I20260924 01:01:01.915899 28510 raft_consensus.cc:2454] T ae853c1bf9b5480eb8b46979dc982bb9 P 155e52068f764292995bd7c462547359 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf2354afa699432899105131c2d64759 in term 1.
I20260924 01:01:01.916174 28707 leader_election.cc:302] T ae853c1bf9b5480eb8b46979dc982bb9 P bf2354afa699432899105131c2d64759 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bf2354afa699432899105131c2d64759, c456b4fca4f64ee6b7c06aa6abc06afc; no voters:
I20260924 01:01:01.916478 28841 raft_consensus.cc:2781] T ae853c1bf9b5480eb8b46979dc982bb9 P bf2354afa699432899105131c2d64759 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 01:01:01.916929 28841 raft_consensus.cc:686] T ae853c1bf9b5480eb8b46979dc982bb9 P bf2354afa699432899105131c2d64759 [term 1 LEADER]: Becoming Leader. State: Replica: bf2354afa699432899105131c2d64759, State: Running, Role: LEADER
I20260924 01:01:01.917492 28841 consensus_queue.cc:237] T ae853c1bf9b5480eb8b46979dc982bb9 P bf2354afa699432899105131c2d64759 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf2354afa699432899105131c2d64759" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 41519 } } peers { permanent_uuid: "c456b4fca4f64ee6b7c06aa6abc06afc" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 34059 } } peers { permanent_uuid: "155e52068f764292995bd7c462547359" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 34249 } }
I20260924 01:01:01.922349 28368 catalog_manager.cc:5495] T ae853c1bf9b5480eb8b46979dc982bb9 P bf2354afa699432899105131c2d64759 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf2354afa699432899105131c2d64759 (127.8.64.131). New cstate: current_term: 1 leader_uuid: "bf2354afa699432899105131c2d64759" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf2354afa699432899105131c2d64759" member_type: VOTER last_known_addr { host: "127.8.64.131" port: 41519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c456b4fca4f64ee6b7c06aa6abc06afc" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 34059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "155e52068f764292995bd7c462547359" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 34249 } health_report { overall_health: UNKNOWN } } }
I20260924 01:01:01.943378 28829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:01.937771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58114 (local address 127.8.64.131:41519)
0924 01:01:01.937988 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:01.937992 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:01.938010 (+ 18us) server_negotiation.cc:407] Connection header received
0924 01:01:01.938187 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:01.938191 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:01.938266 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:01.938469 (+ 203us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 01:01:01.938909 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:01.939686 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:01.941121 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:01.941613 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:01.941718 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:01.942207 (+ 489us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 01:01:01.942805 (+ 598us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 01:01:01.942926 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:01.943030 (+ 104us) server_negotiation.cc:299] Negotiation successful
0924 01:01:01.943089 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260924 01:01:01.951238 28510 raft_consensus.cc:1261] T ae853c1bf9b5480eb8b46979dc982bb9 P 155e52068f764292995bd7c462547359 [term 1 FOLLOWER]: Refusing update from remote peer bf2354afa699432899105131c2d64759: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 01:01:01.951236 28641 raft_consensus.cc:1261] T ae853c1bf9b5480eb8b46979dc982bb9 P c456b4fca4f64ee6b7c06aa6abc06afc [term 1 FOLLOWER]: Refusing update from remote peer bf2354afa699432899105131c2d64759: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 01:01:01.952188 28844 consensus_queue.cc:1035] T ae853c1bf9b5480eb8b46979dc982bb9 P bf2354afa699432899105131c2d64759 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c456b4fca4f64ee6b7c06aa6abc06afc" member_type: VOTER last_known_addr { host: "127.8.64.130" port: 34059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 01:01:01.952560 28841 consensus_queue.cc:1035] T ae853c1bf9b5480eb8b46979dc982bb9 P bf2354afa699432899105131c2d64759 [LEADER]: Connected to new peer: Peer: permanent_uuid: "155e52068f764292995bd7c462547359" member_type: VOTER last_known_addr { host: "127.8.64.129" port: 34249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 01:01:01.964370 28848 mvcc.cc:205] Tried to move back new op lower bound from 7332706967342223360 to 7332706967219032064. Current Snapshot: MvccSnapshot[applied={T|T < 7332706967342223360}]
I20260924 01:01:01.975879 28369 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 01:01:01.976223 28369 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 01:01:01.979982 28369 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 004909555a8f4b8ebebbdf4c30dbe5d2: Sending DeleteTablet for 3 replicas of tablet ae853c1bf9b5480eb8b46979dc982bb9
I20260924 01:01:01.981127 28751 tablet_service.cc:1514] Processing DeleteTablet for tablet ae853c1bf9b5480eb8b46979dc982bb9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 01:01:01 UTC) from {username='slave'} at 127.0.0.1:58110
I20260924 01:01:01.981248 28621 tablet_service.cc:1514] Processing DeleteTablet for tablet ae853c1bf9b5480eb8b46979dc982bb9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 01:01:01 UTC) from {username='slave'} at 127.0.0.1:52680
I20260924 01:01:01.982110 28857 tablet_replica.cc:330] T ae853c1bf9b5480eb8b46979dc982bb9 P bf2354afa699432899105131c2d64759: stopping tablet replica
I20260924 01:01:01.982052 28490 tablet_service.cc:1514] Processing DeleteTablet for tablet ae853c1bf9b5480eb8b46979dc982bb9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 01:01:01 UTC) from {username='slave'} at 127.0.0.1:45082
I20260924 01:01:01.982793 28857 raft_consensus.cc:2229] T ae853c1bf9b5480eb8b46979dc982bb9 P bf2354afa699432899105131c2d64759 [term 1 LEADER]: Raft consensus shutting down.
I20260924 01:01:01.983613 28857 raft_consensus.cc:2258] T ae853c1bf9b5480eb8b46979dc982bb9 P bf2354afa699432899105131c2d64759 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 01:01:01.983970 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 28428
I20260924 01:01:01.986094 28857 ts_tablet_manager.cc:1905] T ae853c1bf9b5480eb8b46979dc982bb9 P bf2354afa699432899105131c2d64759: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 01:01:01.993625 28859 tablet_replica.cc:330] T ae853c1bf9b5480eb8b46979dc982bb9 P 155e52068f764292995bd7c462547359: stopping tablet replica
I20260924 01:01:01.994210 28859 raft_consensus.cc:2229] T ae853c1bf9b5480eb8b46979dc982bb9 P 155e52068f764292995bd7c462547359 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 01:01:01.994855 28859 raft_consensus.cc:2258] T ae853c1bf9b5480eb8b46979dc982bb9 P 155e52068f764292995bd7c462547359 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 01:01:01.995481 28857 ts_tablet_manager.cc:1918] T ae853c1bf9b5480eb8b46979dc982bb9 P bf2354afa699432899105131c2d64759: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 01:01:01.995744 28857 log.cc:1196] T ae853c1bf9b5480eb8b46979dc982bb9 P bf2354afa699432899105131c2d64759: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-2/wal/wals/ae853c1bf9b5480eb8b46979dc982bb9
I20260924 01:01:01.996505 28857 ts_tablet_manager.cc:1939] T ae853c1bf9b5480eb8b46979dc982bb9 P bf2354afa699432899105131c2d64759: Deleting consensus metadata
I20260924 01:01:01.996721 28858 tablet_replica.cc:330] T ae853c1bf9b5480eb8b46979dc982bb9 P c456b4fca4f64ee6b7c06aa6abc06afc: stopping tablet replica
I20260924 01:01:01.996945 28859 ts_tablet_manager.cc:1905] T ae853c1bf9b5480eb8b46979dc982bb9 P 155e52068f764292995bd7c462547359: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 01:01:01.997264 28858 raft_consensus.cc:2229] T ae853c1bf9b5480eb8b46979dc982bb9 P c456b4fca4f64ee6b7c06aa6abc06afc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 01:01:01.997792 28858 raft_consensus.cc:2258] T ae853c1bf9b5480eb8b46979dc982bb9 P c456b4fca4f64ee6b7c06aa6abc06afc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 01:01:01.998399 28356 catalog_manager.cc:4841] TS bf2354afa699432899105131c2d64759 (127.8.64.131:41519): tablet ae853c1bf9b5480eb8b46979dc982bb9 (table test-table [id=902a0555f58249188778c0732185089c]) successfully deleted
I20260924 01:01:02.000046 28858 ts_tablet_manager.cc:1905] T ae853c1bf9b5480eb8b46979dc982bb9 P c456b4fca4f64ee6b7c06aa6abc06afc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 01:01:02.005682 28858 ts_tablet_manager.cc:1918] T ae853c1bf9b5480eb8b46979dc982bb9 P c456b4fca4f64ee6b7c06aa6abc06afc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 01:01:02.005704 28859 ts_tablet_manager.cc:1918] T ae853c1bf9b5480eb8b46979dc982bb9 P 155e52068f764292995bd7c462547359: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 01:01:02.005844 28859 log.cc:1196] T ae853c1bf9b5480eb8b46979dc982bb9 P 155e52068f764292995bd7c462547359: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-0/wal/wals/ae853c1bf9b5480eb8b46979dc982bb9
I20260924 01:01:02.005934 28858 log.cc:1196] T ae853c1bf9b5480eb8b46979dc982bb9 P c456b4fca4f64ee6b7c06aa6abc06afc: Deleting WAL directory at /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790211343099518-8450-0/minicluster-data/ts-1/wal/wals/ae853c1bf9b5480eb8b46979dc982bb9
I20260924 01:01:02.006485 28859 ts_tablet_manager.cc:1939] T ae853c1bf9b5480eb8b46979dc982bb9 P 155e52068f764292995bd7c462547359: Deleting consensus metadata
I20260924 01:01:02.006534 28858 ts_tablet_manager.cc:1939] T ae853c1bf9b5480eb8b46979dc982bb9 P c456b4fca4f64ee6b7c06aa6abc06afc: Deleting consensus metadata
I20260924 01:01:02.008255 28356 catalog_manager.cc:4841] TS 155e52068f764292995bd7c462547359 (127.8.64.129:34249): tablet ae853c1bf9b5480eb8b46979dc982bb9 (table test-table [id=902a0555f58249188778c0732185089c]) successfully deleted
I20260924 01:01:02.008536 28356 catalog_manager.cc:4841] TS c456b4fca4f64ee6b7c06aa6abc06afc (127.8.64.130:34059): tablet ae853c1bf9b5480eb8b46979dc982bb9 (table test-table [id=902a0555f58249188778c0732185089c]) successfully deleted
I20260924 01:01:02.011096 28833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:01.985513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45094 (local address 127.8.64.129:34249)
0924 01:01:01.985736 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:01.985745 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:02.002884 (+ 17139us) server_negotiation.cc:407] Connection header received
0924 01:01:02.003206 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:02.003214 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:02.003314 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:02.003413 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:02.004042 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:02.005240 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:02.007233 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:02.007903 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:02.008000 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:02.009435 (+ 1435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:02.009447 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:02.009464 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:01:02.010195 (+ 731us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:02.010253 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:01:02.010284 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:02.010453 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:02.010625 (+ 172us) server_negotiation.cc:299] Negotiation successful
0924 01:01:02.010735 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260924 01:01:02.011929 28550 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 01:01:02.101697 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 28428
I20260924 01:01:02.119875 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 28559
I20260924 01:01:02.127787 28831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:02.120718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52684 (local address 127.8.64.130:34059)
0924 01:01:02.120938 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:02.120945 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:02.120969 (+ 24us) server_negotiation.cc:407] Connection header received
0924 01:01:02.121057 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:02.121063 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:02.121178 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:02.121290 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:02.121765 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:02.122904 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:02.124203 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:02.124846 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:02.125000 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:02.126099 (+ 1099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:02.126111 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:02.126132 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:01:02.126960 (+ 828us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:02.127027 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:01:02.127052 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:02.127167 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:02.127321 (+ 154us) server_negotiation.cc:299] Negotiation successful
0924 01:01:02.127444 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260924 01:01:02.128463 28681 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 01:01:02.228741 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 28559
I20260924 01:01:02.246232 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 28689
I20260924 01:01:02.254467 28829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:02.247093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58122 (local address 127.8.64.131:41519)
0924 01:01:02.247313 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:02.247319 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:02.247342 (+ 23us) server_negotiation.cc:407] Connection header received
0924 01:01:02.247432 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:02.247437 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:02.247563 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:02.247683 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:02.248030 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:02.249251 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:02.250733 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:02.251452 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:02.251600 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:02.252695 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:02.252705 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:02.252725 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:01:02.253566 (+ 841us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:02.253626 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:01:02.253653 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:02.253769 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:02.253937 (+ 168us) server_negotiation.cc:299] Negotiation successful
0924 01:01:02.254039 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260924 01:01:02.255278 28811 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 01:01:02.376660 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 28689
I20260924 01:01:02.396467 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 28338
I20260924 01:01:02.397552 28399 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 01:01:02.506116 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 28338
2026-09-24T01:01:02Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2723 ms)
[----------] 6 tests from EncryptionPolicyTest (21627 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790211343099518-8450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 01:01:02 dist-test-slave-mmhs krb5kdc[28867](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 01:01:02 dist-test-slave-mmhs krb5kdc[28867](info): set up 2 sockets
Sep 24 01:01:02 dist-test-slave-mmhs krb5kdc[28867](info): commencing operation
krb5kdc: starting...
W20260924 01:01:03.628463 8450 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.001s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Sep 24 01:01:03 dist-test-slave-mmhs krb5kdc[28867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T01:01:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T01:01:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 01:01:03.906251 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37143
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37143
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 01:01:04.238149 28880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:01:04.238526 28880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:01:04.238693 28880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:01:04.245800 28880 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 01:01:04.245957 28880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:01:04.246070 28880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:01:04.246181 28880 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 01:01:04.246271 28880 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 01:01:04.256201 28880 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37143
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:01:04.257579 28880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:01:04.258893 28880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:01:04.270634 28885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:04.270781 28888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:04.270740 28886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:01:04.271821 28880 server_base.cc:1034] running on GCE node
I20260924 01:01:04.272576 28880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:01:04.273940 28880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:01:04.275132 28880 hybrid_clock.cc:648] HybridClock initialized: now 1790211664275076 us; error 48 us; skew 500 ppm
I20260924 01:01:04.275476 28880 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 01:01:04 dist-test-slave-mmhs krb5kdc[28867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211664, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 01:01:04.284986 28880 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 01:01:04.286572 28880 webserver.cc:460] Webserver started at http://127.0.0.1:43577/ using document root <none> and password file <none>
I20260924 01:01:04.287249 28880 fs_manager.cc:359] Metadata directory not provided
I20260924 01:01:04.287348 28880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:01:04.287621 28880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:01:04.290608 28880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "bd806ccd5baa4c8ea0bd20cbef641290"
format_stamp: "Formatted at 2026-09-24 01:01:04 on dist-test-slave-mmhs"
server_key: "98be2cae66277413310ed1d6279ffdaf"
server_key_iv: "919551c31c608cb2dcaf533f1de51af9"
server_key_version: "encryptionkey@0"
I20260924 01:01:04.291312 28880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "bd806ccd5baa4c8ea0bd20cbef641290"
format_stamp: "Formatted at 2026-09-24 01:01:04 on dist-test-slave-mmhs"
server_key: "98be2cae66277413310ed1d6279ffdaf"
server_key_iv: "919551c31c608cb2dcaf533f1de51af9"
server_key_version: "encryptionkey@0"
I20260924 01:01:04.296501 28880 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260924 01:01:04.299975 28895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:01:04.301272 28880 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260924 01:01:04.301440 28880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "bd806ccd5baa4c8ea0bd20cbef641290"
format_stamp: "Formatted at 2026-09-24 01:01:04 on dist-test-slave-mmhs"
server_key: "98be2cae66277413310ed1d6279ffdaf"
server_key_iv: "919551c31c608cb2dcaf533f1de51af9"
server_key_version: "encryptionkey@0"
I20260924 01:01:04.301616 28880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:01:04.344383 28880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:01:04.349063 28880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:01:04.349390 28880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:01:04.369235 28880 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37143
I20260924 01:01:04.369257 28946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37143 every 8 connection(s)
I20260924 01:01:04.370687 28880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 01:01:04.373431 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 28880
I20260924 01:01:04.373662 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 01:01:04.374514 8450 external_mini_cluster.cc:1428] Setting key b29406844c0d5e391b24fbfc0db5d785
I20260924 01:01:04.375068 28947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:01:04.387751 28947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd806ccd5baa4c8ea0bd20cbef641290: Bootstrap starting.
Sep 24 01:01:04 dist-test-slave-mmhs krb5kdc[28867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260924 01:01:04.391001 28947 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd806ccd5baa4c8ea0bd20cbef641290: Neither blocks nor log segments found. Creating new log.
I20260924 01:01:04.392300 28947 log.cc:824] T 00000000000000000000000000000000 P bd806ccd5baa4c8ea0bd20cbef641290: Log is configured to *not* fsync() on all Append() calls
I20260924 01:01:04.395396 28947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd806ccd5baa4c8ea0bd20cbef641290: No bootstrap required, opened a new log
I20260924 01:01:04.398276 28950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:04.376593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57459 (local address 127.0.0.1:37143)
0924 01:01:04.377489 (+ 896us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:04.377501 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:04.377543 (+ 42us) server_negotiation.cc:407] Connection header received
0924 01:01:04.379886 (+ 2343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:04.379910 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:04.380318 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:04.380763 (+ 445us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:04.382360 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:04.383644 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:04.384975 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:04.385688 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:04.385904 (+ 216us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:04.390041 (+ 4137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:04.390049 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:04.390063 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:01:04.390104 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:04.393879 (+ 3775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:04.394796 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:04.394802 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:04.394808 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:04.394979 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:04.395440 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:04.395444 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:04.395446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:04.396018 (+ 572us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:04.396360 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:04.396934 (+ 574us) server_negotiation.cc:299] Negotiation successful
0924 01:01:04.397244 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":660,"thread_start_us":442,"threads_started":1}
I20260924 01:01:04.402486 8450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 01:01:04.403957 28947 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd806ccd5baa4c8ea0bd20cbef641290 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd806ccd5baa4c8ea0bd20cbef641290" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37143 } }
I20260924 01:01:04.404351 28947 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd806ccd5baa4c8ea0bd20cbef641290 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 01:01:04.404465 28947 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd806ccd5baa4c8ea0bd20cbef641290 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd806ccd5baa4c8ea0bd20cbef641290, State: Initialized, Role: FOLLOWER
I20260924 01:01:04.405169 28947 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd806ccd5baa4c8ea0bd20cbef641290 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd806ccd5baa4c8ea0bd20cbef641290" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37143 } }
I20260924 01:01:04.405426 28947 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd806ccd5baa4c8ea0bd20cbef641290 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 01:01:04.405557 28947 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd806ccd5baa4c8ea0bd20cbef641290 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 01:01:04.405731 28947 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd806ccd5baa4c8ea0bd20cbef641290 [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:01:04.408684 28947 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd806ccd5baa4c8ea0bd20cbef641290 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd806ccd5baa4c8ea0bd20cbef641290" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37143 } }
I20260924 01:01:04.409335 28947 leader_election.cc:302] T 00000000000000000000000000000000 P bd806ccd5baa4c8ea0bd20cbef641290 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd806ccd5baa4c8ea0bd20cbef641290; no voters:
I20260924 01:01:04.410015 28947 leader_election.cc:288] T 00000000000000000000000000000000 P bd806ccd5baa4c8ea0bd20cbef641290 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 01:01:04.410221 28957 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd806ccd5baa4c8ea0bd20cbef641290 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 01:01:04.410938 28957 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd806ccd5baa4c8ea0bd20cbef641290 [term 1 LEADER]: Becoming Leader. State: Replica: bd806ccd5baa4c8ea0bd20cbef641290, State: Running, Role: LEADER
I20260924 01:01:04.411720 28957 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd806ccd5baa4c8ea0bd20cbef641290 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd806ccd5baa4c8ea0bd20cbef641290" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37143 } }
I20260924 01:01:04.413453 28947 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd806ccd5baa4c8ea0bd20cbef641290 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 01:01:04.414608 28959 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd806ccd5baa4c8ea0bd20cbef641290 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bd806ccd5baa4c8ea0bd20cbef641290" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd806ccd5baa4c8ea0bd20cbef641290" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37143 } } }
I20260924 01:01:04.415019 28959 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd806ccd5baa4c8ea0bd20cbef641290 [sys.catalog]: This master's current role is: LEADER
I20260924 01:01:04.419395 28958 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd806ccd5baa4c8ea0bd20cbef641290 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd806ccd5baa4c8ea0bd20cbef641290. Latest consensus state: current_term: 1 leader_uuid: "bd806ccd5baa4c8ea0bd20cbef641290" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd806ccd5baa4c8ea0bd20cbef641290" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37143 } } }
I20260924 01:01:04.419835 28958 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd806ccd5baa4c8ea0bd20cbef641290 [sys.catalog]: This master's current role is: LEADER
I20260924 01:01:04.422675 28962 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 01:01:04.428254 28950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:04.408261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52683 (local address 127.0.0.1:37143)
0924 01:01:04.408423 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:04.408427 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:04.409028 (+ 601us) server_negotiation.cc:407] Connection header received
0924 01:01:04.409102 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:04.409107 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:04.409185 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:04.409294 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:04.411441 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:04.412674 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:04.414699 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:04.415497 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:04.415665 (+ 168us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:04.420008 (+ 4343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:04.420018 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:04.420026 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:01:04.420079 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:04.424165 (+ 4086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:04.425279 (+ 1114us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:04.425285 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:04.425289 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:04.425501 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:04.426156 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:04.426161 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:04.426164 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:04.426637 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:04.426933 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:04.427653 (+ 720us) server_negotiation.cc:299] Negotiation successful
0924 01:01:04.427853 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260924 01:01:04.430092 28962 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 01:01:04.442520 28962 catalog_manager.cc:1347] Generated new cluster ID: 101fc7ecd0334476af66135d3f659b0c
I20260924 01:01:04.442663 28962 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 01:01:04.466395 28962 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 01:01:04.467787 28962 catalog_manager.cc:1504] Loading token signing keys...
I20260924 01:01:04.477762 28962 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bd806ccd5baa4c8ea0bd20cbef641290: Generated new TSK 0
I20260924 01:01:04.478703 28962 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 01:01:04.488420 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 28880
I20260924 01:01:04.489776 28942 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 01:01:04.584081 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 28880
W20260924 01:01:04.606241 28865 connection.cc:535] client connection to 127.0.0.1:37143 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37143): Connection reset by peer (error 104)
2026-09-24T01:01:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2076 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790211343099518-8450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 01:01:04 dist-test-slave-mmhs krb5kdc[28979](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 01:01:04 dist-test-slave-mmhs krb5kdc[28979](info): set up 2 sockets
Sep 24 01:01:04 dist-test-slave-mmhs krb5kdc[28979](info): commencing operation
krb5kdc: starting...
W20260924 01:01:05.708196 8450 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.001s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Sep 24 01:01:05 dist-test-slave-mmhs krb5kdc[28979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T01:01:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T01:01:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 01:01:05.973891 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43521
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43521
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 01:01:06.303296 28992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:01:06.303560 28992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:01:06.303717 28992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:01:06.310910 28992 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 01:01:06.311018 28992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:01:06.311108 28992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:01:06.311208 28992 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 01:01:06.311272 28992 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 01:01:06.320977 28992 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43521
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:01:06.322214 28992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:01:06.323504 28992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:01:06.335150 29000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:06.335227 28997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:06.335698 28998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:01:06.335981 28992 server_base.cc:1034] running on GCE node
I20260924 01:01:06.336810 28992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:01:06.338171 28992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:01:06.339424 28992 hybrid_clock.cc:648] HybridClock initialized: now 1790211666339370 us; error 41 us; skew 500 ppm
I20260924 01:01:06.339752 28992 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 01:01:06 dist-test-slave-mmhs krb5kdc[28979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211666, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 01:01:06.342885 28992 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 01:01:06.344388 28992 webserver.cc:460] Webserver started at http://127.0.0.1:46235/ using document root <none> and password file <none>
I20260924 01:01:06.345039 28992 fs_manager.cc:359] Metadata directory not provided
I20260924 01:01:06.345139 28992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:01:06.345387 28992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:01:06.348332 28992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "2b357f5ae5184178b5a201b22ff7058b"
format_stamp: "Formatted at 2026-09-24 01:01:06 on dist-test-slave-mmhs"
server_key: "22dbf512cdddd163638c0713daa96b16"
server_key_iv: "afee7ee716261ecaf5ddab4f16f0d31b"
server_key_version: "encryptionkey@0"
I20260924 01:01:06.349103 28992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "2b357f5ae5184178b5a201b22ff7058b"
format_stamp: "Formatted at 2026-09-24 01:01:06 on dist-test-slave-mmhs"
server_key: "22dbf512cdddd163638c0713daa96b16"
server_key_iv: "afee7ee716261ecaf5ddab4f16f0d31b"
server_key_version: "encryptionkey@0"
I20260924 01:01:06.354142 28992 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260924 01:01:06.357519 29007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:01:06.358968 28992 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 01:01:06.359094 28992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "2b357f5ae5184178b5a201b22ff7058b"
format_stamp: "Formatted at 2026-09-24 01:01:06 on dist-test-slave-mmhs"
server_key: "22dbf512cdddd163638c0713daa96b16"
server_key_iv: "afee7ee716261ecaf5ddab4f16f0d31b"
server_key_version: "encryptionkey@0"
I20260924 01:01:06.359248 28992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:01:06.375403 28992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:01:06.379793 28992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:01:06.380026 28992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:01:06.398677 28992 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43521
I20260924 01:01:06.398716 29058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43521 every 8 connection(s)
I20260924 01:01:06.400187 28992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 01:01:06.403895 29059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:01:06.409861 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 28992
I20260924 01:01:06.410063 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 01:01:06.410594 8450 external_mini_cluster.cc:1428] Setting key 08f1df38e7f7fb4949a62d39f083413c
I20260924 01:01:06.414866 29059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b357f5ae5184178b5a201b22ff7058b: Bootstrap starting.
I20260924 01:01:06.418350 29059 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b357f5ae5184178b5a201b22ff7058b: Neither blocks nor log segments found. Creating new log.
I20260924 01:01:06.419612 29059 log.cc:824] T 00000000000000000000000000000000 P 2b357f5ae5184178b5a201b22ff7058b: Log is configured to *not* fsync() on all Append() calls
I20260924 01:01:06.422868 29059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b357f5ae5184178b5a201b22ff7058b: No bootstrap required, opened a new log
I20260924 01:01:06.424515 29062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:06.412509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40081 (local address 127.0.0.1:43521)
0924 01:01:06.413332 (+ 823us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:06.413343 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:06.413384 (+ 41us) server_negotiation.cc:407] Connection header received
0924 01:01:06.415252 (+ 1868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:06.415274 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:06.415663 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:06.415975 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:06.416609 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:06.417879 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:06.419821 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:06.420853 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:06.421030 (+ 177us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:06.422239 (+ 1209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:06.422247 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:06.422260 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:01:06.422840 (+ 580us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:06.422911 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:01:06.422936 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:06.423107 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:06.423389 (+ 282us) server_negotiation.cc:299] Negotiation successful
0924 01:01:06.423579 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":396,"threads_started":1}
I20260924 01:01:06.428259 8450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 01:01:06.431028 29059 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b357f5ae5184178b5a201b22ff7058b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b357f5ae5184178b5a201b22ff7058b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43521 } }
I20260924 01:01:06.431401 29059 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b357f5ae5184178b5a201b22ff7058b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 01:01:06.431525 29059 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b357f5ae5184178b5a201b22ff7058b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b357f5ae5184178b5a201b22ff7058b, State: Initialized, Role: FOLLOWER
I20260924 01:01:06.432137 29059 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b357f5ae5184178b5a201b22ff7058b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b357f5ae5184178b5a201b22ff7058b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43521 } }
I20260924 01:01:06.432406 29059 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b357f5ae5184178b5a201b22ff7058b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 01:01:06.432585 29059 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b357f5ae5184178b5a201b22ff7058b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 01:01:06.432788 29059 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b357f5ae5184178b5a201b22ff7058b [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:01:06.436152 29059 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b357f5ae5184178b5a201b22ff7058b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b357f5ae5184178b5a201b22ff7058b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43521 } }
I20260924 01:01:06.436753 29059 leader_election.cc:302] T 00000000000000000000000000000000 P 2b357f5ae5184178b5a201b22ff7058b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b357f5ae5184178b5a201b22ff7058b; no voters:
I20260924 01:01:06.437582 29059 leader_election.cc:288] T 00000000000000000000000000000000 P 2b357f5ae5184178b5a201b22ff7058b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 01:01:06.437706 29069 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b357f5ae5184178b5a201b22ff7058b [term 1 FOLLOWER]: Leader election won for term 1
I20260924 01:01:06.440169 29062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:06.432552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43247 (local address 127.0.0.1:43521)
0924 01:01:06.432759 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:06.432767 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:06.433251 (+ 484us) server_negotiation.cc:407] Connection header received
0924 01:01:06.433435 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:06.433438 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:06.433496 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:06.433605 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:06.434244 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:06.435084 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:06.436972 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:06.437450 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:06.437616 (+ 166us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:06.439012 (+ 1396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:06.439018 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:06.439023 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:01:06.439043 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:06.439087 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:01:06.439104 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:06.439229 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:06.439632 (+ 403us) server_negotiation.cc:299] Negotiation successful
0924 01:01:06.439691 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260924 01:01:06.440352 29059 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b357f5ae5184178b5a201b22ff7058b [sys.catalog]: configured and running, proceeding with master startup.
I20260924 01:01:06.439991 29069 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b357f5ae5184178b5a201b22ff7058b [term 1 LEADER]: Becoming Leader. State: Replica: 2b357f5ae5184178b5a201b22ff7058b, State: Running, Role: LEADER
I20260924 01:01:06.441476 29069 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b357f5ae5184178b5a201b22ff7058b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b357f5ae5184178b5a201b22ff7058b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43521 } }
I20260924 01:01:06.445410 29071 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b357f5ae5184178b5a201b22ff7058b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b357f5ae5184178b5a201b22ff7058b. Latest consensus state: current_term: 1 leader_uuid: "2b357f5ae5184178b5a201b22ff7058b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b357f5ae5184178b5a201b22ff7058b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43521 } } }
I20260924 01:01:06.445530 29070 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b357f5ae5184178b5a201b22ff7058b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b357f5ae5184178b5a201b22ff7058b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b357f5ae5184178b5a201b22ff7058b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43521 } } }
I20260924 01:01:06.445968 29070 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b357f5ae5184178b5a201b22ff7058b [sys.catalog]: This master's current role is: LEADER
I20260924 01:01:06.445849 29071 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b357f5ae5184178b5a201b22ff7058b [sys.catalog]: This master's current role is: LEADER
I20260924 01:01:06.448901 29080 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 01:01:06.454989 29080 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 01:01:06.464002 29080 catalog_manager.cc:1347] Generated new cluster ID: 36f4bd1fa811458d980fd8674603051e
I20260924 01:01:06.464131 29080 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 01:01:06.490862 29080 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 01:01:06.492223 29080 catalog_manager.cc:1504] Loading token signing keys...
I20260924 01:01:06.508643 29080 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b357f5ae5184178b5a201b22ff7058b: Generated new TSK 0
I20260924 01:01:06.509927 29080 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 01:01:06.521299 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 28992
W20260924 01:01:06.524524 29054 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:40081
W20260924 01:01:06.525394 8450 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 28992: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260924 01:01:06.525596 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 28992
W20260924 01:01:06.547832 28977 connection.cc:535] client connection to 127.0.0.1:43521 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43521): Connection reset by peer (error 104)
2026-09-24T01:01:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1941 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790211343099518-8450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 01:01:06 dist-test-slave-mmhs krb5kdc[29090](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 01:01:06 dist-test-slave-mmhs krb5kdc[29090](info): set up 2 sockets
Sep 24 01:01:06 dist-test-slave-mmhs krb5kdc[29090](info): commencing operation
krb5kdc: starting...
W20260924 01:01:07.644249 8450 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Sep 24 01:01:07 dist-test-slave-mmhs krb5kdc[29090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T01:01:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T01:01:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 01:01:07.915900 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44801
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44801
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 01:01:08.247258 29103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:01:08.247572 29103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:01:08.247736 29103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:01:08.254900 29103 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 01:01:08.255043 29103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:01:08.255141 29103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:01:08.255246 29103 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 01:01:08.255321 29103 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 01:01:08.264998 29103 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44801
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:01:08.266355 29103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:01:08.267627 29103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:01:08.279307 29109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:08.279429 29108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:08.279400 29111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:01:08.280318 29103 server_base.cc:1034] running on GCE node
I20260924 01:01:08.281005 29103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:01:08.282358 29103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:01:08.283617 29103 hybrid_clock.cc:648] HybridClock initialized: now 1790211668283576 us; error 40 us; skew 500 ppm
I20260924 01:01:08.283962 29103 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 01:01:08 dist-test-slave-mmhs krb5kdc[29090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211668, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 01:01:08.287119 29103 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 01:01:08.288705 29103 webserver.cc:460] Webserver started at http://127.0.0.1:35763/ using document root <none> and password file <none>
I20260924 01:01:08.289378 29103 fs_manager.cc:359] Metadata directory not provided
I20260924 01:01:08.289505 29103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:01:08.289812 29103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:01:08.292783 29103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "fb8379cad23a4476a81a2602f9d5b47c"
format_stamp: "Formatted at 2026-09-24 01:01:08 on dist-test-slave-mmhs"
server_key: "d96906f796610ba95c53990b5690091c"
server_key_iv: "5ecad6693bc20e5df868d3da9a78b240"
server_key_version: "encryptionkey@0"
I20260924 01:01:08.293566 29103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "fb8379cad23a4476a81a2602f9d5b47c"
format_stamp: "Formatted at 2026-09-24 01:01:08 on dist-test-slave-mmhs"
server_key: "d96906f796610ba95c53990b5690091c"
server_key_iv: "5ecad6693bc20e5df868d3da9a78b240"
server_key_version: "encryptionkey@0"
I20260924 01:01:08.298753 29103 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260924 01:01:08.302110 29118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:01:08.303450 29103 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 01:01:08.303627 29103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "fb8379cad23a4476a81a2602f9d5b47c"
format_stamp: "Formatted at 2026-09-24 01:01:08 on dist-test-slave-mmhs"
server_key: "d96906f796610ba95c53990b5690091c"
server_key_iv: "5ecad6693bc20e5df868d3da9a78b240"
server_key_version: "encryptionkey@0"
I20260924 01:01:08.303820 29103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:01:08.309976 29103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:01:08.313994 29103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:01:08.314384 29103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:01:08.332573 29103 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44801
I20260924 01:01:08.332659 29169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44801 every 8 connection(s)
I20260924 01:01:08.333945 29103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 01:01:08.337483 29170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:01:08.342489 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 29103
I20260924 01:01:08.342670 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 01:01:08.343175 8450 external_mini_cluster.cc:1428] Setting key f3432cddbc4b21837679b3217cba2336
I20260924 01:01:08.348731 29170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb8379cad23a4476a81a2602f9d5b47c: Bootstrap starting.
I20260924 01:01:08.350977 29173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:08.344962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45217 (local address 127.0.0.1:44801)
0924 01:01:08.345774 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:08.345785 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:08.345827 (+ 42us) server_negotiation.cc:407] Connection header received
0924 01:01:08.347591 (+ 1764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:08.347612 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:08.347938 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:08.348242 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:08.349307 (+ 1065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:08.349313 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:08.349328 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:01:08.349347 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:08.349400 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:01:08.349420 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:08.349618 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:08.349727 (+ 109us) server_negotiation.cc:299] Negotiation successful
0924 01:01:08.349940 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":389,"threads_started":1}
I20260924 01:01:08.352408 29170 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb8379cad23a4476a81a2602f9d5b47c: Neither blocks nor log segments found. Creating new log.
I20260924 01:01:08.353598 29170 log.cc:824] T 00000000000000000000000000000000 P fb8379cad23a4476a81a2602f9d5b47c: Log is configured to *not* fsync() on all Append() calls
I20260924 01:01:08.354689 8450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 01:01:08.356812 29170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb8379cad23a4476a81a2602f9d5b47c: No bootstrap required, opened a new log
I20260924 01:01:08.362278 29173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:08.359728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43757 (local address 127.0.0.1:44801)
0924 01:01:08.359959 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:08.359968 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:08.360414 (+ 446us) server_negotiation.cc:407] Connection header received
0924 01:01:08.360540 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:08.360544 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:08.360592 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:08.360692 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:08.361570 (+ 878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:08.361574 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:08.361577 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:01:08.361591 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:08.361625 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:01:08.361640 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:08.361732 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:08.361857 (+ 125us) server_negotiation.cc:299] Negotiation successful
0924 01:01:08.361903 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260924 01:01:08.365980 29170 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb8379cad23a4476a81a2602f9d5b47c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb8379cad23a4476a81a2602f9d5b47c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44801 } }
I20260924 01:01:08.366364 29170 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb8379cad23a4476a81a2602f9d5b47c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 01:01:08.366482 29170 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb8379cad23a4476a81a2602f9d5b47c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb8379cad23a4476a81a2602f9d5b47c, State: Initialized, Role: FOLLOWER
I20260924 01:01:08.367100 29170 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb8379cad23a4476a81a2602f9d5b47c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb8379cad23a4476a81a2602f9d5b47c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44801 } }
I20260924 01:01:08.367347 29170 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb8379cad23a4476a81a2602f9d5b47c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 01:01:08.367491 29170 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb8379cad23a4476a81a2602f9d5b47c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 01:01:08.367683 29170 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb8379cad23a4476a81a2602f9d5b47c [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:01:08.370157 29170 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb8379cad23a4476a81a2602f9d5b47c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb8379cad23a4476a81a2602f9d5b47c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44801 } }
I20260924 01:01:08.370683 29170 leader_election.cc:302] T 00000000000000000000000000000000 P fb8379cad23a4476a81a2602f9d5b47c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb8379cad23a4476a81a2602f9d5b47c; no voters:
I20260924 01:01:08.371253 29170 leader_election.cc:288] T 00000000000000000000000000000000 P fb8379cad23a4476a81a2602f9d5b47c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 01:01:08.371416 29180 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb8379cad23a4476a81a2602f9d5b47c [term 1 FOLLOWER]: Leader election won for term 1
I20260924 01:01:08.372103 29180 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb8379cad23a4476a81a2602f9d5b47c [term 1 LEADER]: Becoming Leader. State: Replica: fb8379cad23a4476a81a2602f9d5b47c, State: Running, Role: LEADER
I20260924 01:01:08.372648 29180 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb8379cad23a4476a81a2602f9d5b47c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb8379cad23a4476a81a2602f9d5b47c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44801 } }
I20260924 01:01:08.373203 29170 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb8379cad23a4476a81a2602f9d5b47c [sys.catalog]: configured and running, proceeding with master startup.
I20260924 01:01:08.375741 29181 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb8379cad23a4476a81a2602f9d5b47c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb8379cad23a4476a81a2602f9d5b47c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb8379cad23a4476a81a2602f9d5b47c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44801 } } }
I20260924 01:01:08.376652 29181 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb8379cad23a4476a81a2602f9d5b47c [sys.catalog]: This master's current role is: LEADER
I20260924 01:01:08.375949 29182 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb8379cad23a4476a81a2602f9d5b47c [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb8379cad23a4476a81a2602f9d5b47c. Latest consensus state: current_term: 1 leader_uuid: "fb8379cad23a4476a81a2602f9d5b47c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb8379cad23a4476a81a2602f9d5b47c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44801 } } }
I20260924 01:01:08.377071 29182 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb8379cad23a4476a81a2602f9d5b47c [sys.catalog]: This master's current role is: LEADER
I20260924 01:01:08.378098 29188 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 01:01:08.383320 29188 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 01:01:08.392043 29188 catalog_manager.cc:1347] Generated new cluster ID: dc88fd64b0f2445882ecbdc76cca232d
I20260924 01:01:08.392181 29188 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 01:01:08.402915 29188 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 01:01:08.403386 29188 catalog_manager.cc:1504] Loading token signing keys...
I20260924 01:01:08.413172 29188 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb8379cad23a4476a81a2602f9d5b47c: Generated new TSK 0
I20260924 01:01:08.414124 29188 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 01:01:08.422258 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 29103
W20260924 01:01:08.423310 29165 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:45217
W20260924 01:01:08.423993 8450 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 29103: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260924 01:01:08.424139 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 29103
2026-09-24T01:01:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1890 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790211343099518-8450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 01:01:08 dist-test-slave-mmhs krb5kdc[29201](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 01:01:08 dist-test-slave-mmhs krb5kdc[29201](info): set up 2 sockets
Sep 24 01:01:08 dist-test-slave-mmhs krb5kdc[29201](info): commencing operation
krb5kdc: starting...
W20260924 01:01:09.528415 8450 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.003s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Sep 24 01:01:09 dist-test-slave-mmhs krb5kdc[29201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211669, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T01:01:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T01:01:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 01:01:09.794713 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33921
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33921
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 01:01:10.120645 29214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:01:10.120911 29214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:01:10.121121 29214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:01:10.128274 29214 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 01:01:10.128373 29214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:01:10.128603 29214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:01:10.128715 29214 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 01:01:10.128813 29214 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 01:01:10.138638 29214 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33921
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:01:10.139931 29214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:01:10.141248 29214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:01:10.153061 29219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:10.153097 29220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:10.153138 29222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:01:10.154088 29214 server_base.cc:1034] running on GCE node
I20260924 01:01:10.154888 29214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:01:10.156236 29214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:01:10.157445 29214 hybrid_clock.cc:648] HybridClock initialized: now 1790211670157404 us; error 49 us; skew 500 ppm
I20260924 01:01:10.157799 29214 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 01:01:10 dist-test-slave-mmhs krb5kdc[29201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211670, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 01:01:10.168432 29214 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 01:01:10.169967 29214 webserver.cc:460] Webserver started at http://127.0.0.1:40369/ using document root <none> and password file <none>
I20260924 01:01:10.170749 29214 fs_manager.cc:359] Metadata directory not provided
I20260924 01:01:10.170889 29214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:01:10.171207 29214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:01:10.174132 29214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "6ae0731cb3744917874a6b9651053691"
format_stamp: "Formatted at 2026-09-24 01:01:10 on dist-test-slave-mmhs"
server_key: "eb8b6bc66f62883eea17ebe13e62ba14"
server_key_iv: "037b39f6700205bcb11b394bf60d0599"
server_key_version: "encryptionkey@0"
I20260924 01:01:10.174942 29214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "6ae0731cb3744917874a6b9651053691"
format_stamp: "Formatted at 2026-09-24 01:01:10 on dist-test-slave-mmhs"
server_key: "eb8b6bc66f62883eea17ebe13e62ba14"
server_key_iv: "037b39f6700205bcb11b394bf60d0599"
server_key_version: "encryptionkey@0"
I20260924 01:01:10.179970 29214 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260924 01:01:10.183260 29229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:01:10.184521 29214 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260924 01:01:10.184698 29214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "6ae0731cb3744917874a6b9651053691"
format_stamp: "Formatted at 2026-09-24 01:01:10 on dist-test-slave-mmhs"
server_key: "eb8b6bc66f62883eea17ebe13e62ba14"
server_key_iv: "037b39f6700205bcb11b394bf60d0599"
server_key_version: "encryptionkey@0"
I20260924 01:01:10.184898 29214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:01:10.201424 29214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:01:10.205397 29214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:01:10.205677 29214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:01:10.224153 29214 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33921
I20260924 01:01:10.224189 29280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33921 every 8 connection(s)
I20260924 01:01:10.225487 29214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 01:01:10.229116 29281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:01:10.229736 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 29214
I20260924 01:01:10.229928 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 01:01:10.230384 8450 external_mini_cluster.cc:1428] Setting key c1a141ec4548a214c03dc1cb1448903e
I20260924 01:01:10.241365 29281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ae0731cb3744917874a6b9651053691: Bootstrap starting.
Sep 24 01:01:10 dist-test-slave-mmhs krb5kdc[29201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211669, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260924 01:01:10.244644 29281 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ae0731cb3744917874a6b9651053691: Neither blocks nor log segments found. Creating new log.
I20260924 01:01:10.245666 29281 log.cc:824] T 00000000000000000000000000000000 P 6ae0731cb3744917874a6b9651053691: Log is configured to *not* fsync() on all Append() calls
I20260924 01:01:10.248486 29281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ae0731cb3744917874a6b9651053691: No bootstrap required, opened a new log
I20260924 01:01:10.253118 29284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:10.231979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36785 (local address 127.0.0.1:33921)
0924 01:01:10.232834 (+ 855us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:10.232853 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:10.233163 (+ 310us) server_negotiation.cc:407] Connection header received
0924 01:01:10.235533 (+ 2370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:10.235554 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:10.235947 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:10.236338 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:10.237805 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:10.239166 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:10.240405 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:10.241040 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:10.241242 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:10.245687 (+ 4445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:10.245695 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:10.245709 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:01:10.245803 (+ 94us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:10.249021 (+ 3218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:10.249707 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:10.249721 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:10.249727 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:10.249978 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:10.250459 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:10.250462 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:10.250464 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:10.251019 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:10.251249 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:10.251699 (+ 450us) server_negotiation.cc:299] Negotiation successful
0924 01:01:10.252071 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":399,"threads_started":1}
I20260924 01:01:10.256743 8450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 01:01:10.257055 29281 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ae0731cb3744917874a6b9651053691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ae0731cb3744917874a6b9651053691" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33921 } }
I20260924 01:01:10.257400 29281 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ae0731cb3744917874a6b9651053691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 01:01:10.257522 29281 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ae0731cb3744917874a6b9651053691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ae0731cb3744917874a6b9651053691, State: Initialized, Role: FOLLOWER
I20260924 01:01:10.258095 29281 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ae0731cb3744917874a6b9651053691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ae0731cb3744917874a6b9651053691" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33921 } }
I20260924 01:01:10.258343 29281 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ae0731cb3744917874a6b9651053691 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 01:01:10.258499 29281 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ae0731cb3744917874a6b9651053691 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 01:01:10.258661 29281 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ae0731cb3744917874a6b9651053691 [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:01:10.261006 29281 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ae0731cb3744917874a6b9651053691 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ae0731cb3744917874a6b9651053691" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33921 } }
I20260924 01:01:10.261685 29281 leader_election.cc:302] T 00000000000000000000000000000000 P 6ae0731cb3744917874a6b9651053691 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ae0731cb3744917874a6b9651053691; no voters:
I20260924 01:01:10.262423 29281 leader_election.cc:288] T 00000000000000000000000000000000 P 6ae0731cb3744917874a6b9651053691 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 01:01:10.262588 29291 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ae0731cb3744917874a6b9651053691 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 01:01:10.263828 29291 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ae0731cb3744917874a6b9651053691 [term 1 LEADER]: Becoming Leader. State: Replica: 6ae0731cb3744917874a6b9651053691, State: Running, Role: LEADER
I20260924 01:01:10.264622 29281 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ae0731cb3744917874a6b9651053691 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 01:01:10.264470 29291 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ae0731cb3744917874a6b9651053691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ae0731cb3744917874a6b9651053691" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33921 } }
I20260924 01:01:10.267880 29292 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ae0731cb3744917874a6b9651053691 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6ae0731cb3744917874a6b9651053691" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ae0731cb3744917874a6b9651053691" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33921 } } }
I20260924 01:01:10.268224 29292 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ae0731cb3744917874a6b9651053691 [sys.catalog]: This master's current role is: LEADER
I20260924 01:01:10.267601 29293 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ae0731cb3744917874a6b9651053691 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ae0731cb3744917874a6b9651053691. Latest consensus state: current_term: 1 leader_uuid: "6ae0731cb3744917874a6b9651053691" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ae0731cb3744917874a6b9651053691" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33921 } } }
I20260924 01:01:10.268945 29293 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ae0731cb3744917874a6b9651053691 [sys.catalog]: This master's current role is: LEADER
I20260924 01:01:10.282001 29284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:10.261113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33911 (local address 127.0.0.1:33921)
0924 01:01:10.261313 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:10.261321 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:10.261881 (+ 560us) server_negotiation.cc:407] Connection header received
0924 01:01:10.262033 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:10.262037 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:10.262120 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:10.262244 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:10.264144 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:10.265295 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:10.267657 (+ 2362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:10.268691 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:10.268827 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:10.273063 (+ 4236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:10.273072 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:10.273080 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:01:10.273131 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:10.277717 (+ 4586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:10.278760 (+ 1043us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:10.278769 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:10.278775 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:10.278991 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:10.279561 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:10.279569 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:10.279575 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:10.280088 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:10.280387 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:10.281115 (+ 728us) server_negotiation.cc:299] Negotiation successful
0924 01:01:10.281427 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260924 01:01:10.282411 29297 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 01:01:10.288486 29297 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 01:01:10.298427 29297 catalog_manager.cc:1347] Generated new cluster ID: 69026bd9b9fb42fa82e298b72f4e99ec
I20260924 01:01:10.298545 29297 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 01:01:10.314170 29297 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 01:01:10.316038 29297 catalog_manager.cc:1504] Loading token signing keys...
I20260924 01:01:10.325968 29297 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ae0731cb3744917874a6b9651053691: Generated new TSK 0
I20260924 01:01:10.327011 29297 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 01:01:10.340969 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 29214
I20260924 01:01:10.342468 29276 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 01:01:10.432917 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 29214
2026-09-24T01:01:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2012 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790211343099518-8450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 01:01:10 dist-test-slave-mmhs krb5kdc[29314](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 01:01:10 dist-test-slave-mmhs krb5kdc[29314](info): set up 2 sockets
Sep 24 01:01:10 dist-test-slave-mmhs krb5kdc[29314](info): commencing operation
krb5kdc: starting...
W20260924 01:01:11.545327 8450 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.000s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Sep 24 01:01:11 dist-test-slave-mmhs krb5kdc[29314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T01:01:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T01:01:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 01:01:11.809769 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33763
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33763
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 01:01:12.145761 29327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:01:12.146077 29327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:01:12.146284 29327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:01:12.153708 29327 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 01:01:12.153854 29327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:01:12.153950 29327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:01:12.154071 29327 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 01:01:12.154167 29327 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 01:01:12.164212 29327 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33763
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:01:12.165556 29327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:01:12.166910 29327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:01:12.179263 29332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:12.179303 29335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:12.179344 29333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:01:12.180608 29327 server_base.cc:1034] running on GCE node
I20260924 01:01:12.181418 29327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:01:12.182988 29327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:01:12.184219 29327 hybrid_clock.cc:648] HybridClock initialized: now 1790211672184184 us; error 52 us; skew 500 ppm
I20260924 01:01:12.184595 29327 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 01:01:12 dist-test-slave-mmhs krb5kdc[29314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211672, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 01:01:12.187724 29327 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 01:01:12.189356 29327 webserver.cc:460] Webserver started at http://127.0.0.1:43511/ using document root <none> and password file <none>
I20260924 01:01:12.190119 29327 fs_manager.cc:359] Metadata directory not provided
I20260924 01:01:12.190258 29327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:01:12.190606 29327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:01:12.193679 29327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "ce0da1f507184ce183421d9b93b39382"
format_stamp: "Formatted at 2026-09-24 01:01:12 on dist-test-slave-mmhs"
server_key: "1343d96d56f6dd7da03af7cdbdfc4011"
server_key_iv: "d976c7b38c53f35ae3e8eeed6d9a7fe0"
server_key_version: "encryptionkey@0"
I20260924 01:01:12.194563 29327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "ce0da1f507184ce183421d9b93b39382"
format_stamp: "Formatted at 2026-09-24 01:01:12 on dist-test-slave-mmhs"
server_key: "1343d96d56f6dd7da03af7cdbdfc4011"
server_key_iv: "d976c7b38c53f35ae3e8eeed6d9a7fe0"
server_key_version: "encryptionkey@0"
I20260924 01:01:12.199865 29327 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260924 01:01:12.203325 29342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:01:12.204962 29327 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260924 01:01:12.205186 29327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "ce0da1f507184ce183421d9b93b39382"
format_stamp: "Formatted at 2026-09-24 01:01:12 on dist-test-slave-mmhs"
server_key: "1343d96d56f6dd7da03af7cdbdfc4011"
server_key_iv: "d976c7b38c53f35ae3e8eeed6d9a7fe0"
server_key_version: "encryptionkey@0"
I20260924 01:01:12.205461 29327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:01:12.223184 29327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:01:12.227662 29327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:01:12.227950 29327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:01:12.246873 29327 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33763
I20260924 01:01:12.246961 29393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33763 every 8 connection(s)
I20260924 01:01:12.248250 29327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 01:01:12.251859 29394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:01:12.255209 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 29327
I20260924 01:01:12.255384 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 01:01:12.255919 8450 external_mini_cluster.cc:1428] Setting key 3969f3477cdcf7578a10dde797d66a3b
I20260924 01:01:12.263981 29394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce0da1f507184ce183421d9b93b39382: Bootstrap starting.
I20260924 01:01:12.267652 29394 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce0da1f507184ce183421d9b93b39382: Neither blocks nor log segments found. Creating new log.
I20260924 01:01:12.268687 29394 log.cc:824] T 00000000000000000000000000000000 P ce0da1f507184ce183421d9b93b39382: Log is configured to *not* fsync() on all Append() calls
I20260924 01:01:12.269785 29397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:12.257658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59223 (local address 127.0.0.1:33763)
0924 01:01:12.258433 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:12.258445 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:12.258486 (+ 41us) server_negotiation.cc:407] Connection header received
0924 01:01:12.260124 (+ 1638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:12.260145 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:12.260528 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:12.260841 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:12.261424 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:12.262793 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:12.264625 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:12.265295 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:12.265597 (+ 302us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:12.267013 (+ 1416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:12.267020 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:12.267033 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:01:12.267605 (+ 572us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:12.267680 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:01:12.267739 (+ 59us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:12.267858 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:12.268137 (+ 279us) server_negotiation.cc:299] Negotiation successful
0924 01:01:12.268405 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":353,"threads_started":1}
I20260924 01:01:12.271559 29394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce0da1f507184ce183421d9b93b39382: No bootstrap required, opened a new log
I20260924 01:01:12.273363 8450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 01:01:12.279922 29394 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce0da1f507184ce183421d9b93b39382 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce0da1f507184ce183421d9b93b39382" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33763 } }
I20260924 01:01:12.280336 29394 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce0da1f507184ce183421d9b93b39382 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 01:01:12.280457 29394 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce0da1f507184ce183421d9b93b39382 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce0da1f507184ce183421d9b93b39382, State: Initialized, Role: FOLLOWER
I20260924 01:01:12.281116 29394 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce0da1f507184ce183421d9b93b39382 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce0da1f507184ce183421d9b93b39382" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33763 } }
I20260924 01:01:12.281378 29394 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce0da1f507184ce183421d9b93b39382 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 01:01:12.281509 29394 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce0da1f507184ce183421d9b93b39382 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 01:01:12.281678 29394 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce0da1f507184ce183421d9b93b39382 [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:01:12.284302 29397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:12.277809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57203 (local address 127.0.0.1:33763)
0924 01:01:12.278024 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:12.278030 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:12.278359 (+ 329us) server_negotiation.cc:407] Connection header received
0924 01:01:12.278556 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:12.278559 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:12.278626 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:12.278710 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:12.279317 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:12.280039 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:12.281885 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:12.282407 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:12.282541 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:12.283599 (+ 1058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:12.283604 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:12.283607 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:01:12.283622 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:12.283661 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:01:12.283675 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:12.283745 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:12.283898 (+ 153us) server_negotiation.cc:299] Negotiation successful
0924 01:01:12.283970 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260924 01:01:12.284794 29394 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce0da1f507184ce183421d9b93b39382 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce0da1f507184ce183421d9b93b39382" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33763 } }
I20260924 01:01:12.285300 29394 leader_election.cc:302] T 00000000000000000000000000000000 P ce0da1f507184ce183421d9b93b39382 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce0da1f507184ce183421d9b93b39382; no voters:
I20260924 01:01:12.285956 29394 leader_election.cc:288] T 00000000000000000000000000000000 P ce0da1f507184ce183421d9b93b39382 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 01:01:12.286059 29404 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce0da1f507184ce183421d9b93b39382 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 01:01:12.286791 29404 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce0da1f507184ce183421d9b93b39382 [term 1 LEADER]: Becoming Leader. State: Replica: ce0da1f507184ce183421d9b93b39382, State: Running, Role: LEADER
I20260924 01:01:12.287444 29404 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce0da1f507184ce183421d9b93b39382 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce0da1f507184ce183421d9b93b39382" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33763 } }
I20260924 01:01:12.288287 29394 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce0da1f507184ce183421d9b93b39382 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 01:01:12.290472 29406 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce0da1f507184ce183421d9b93b39382 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce0da1f507184ce183421d9b93b39382. Latest consensus state: current_term: 1 leader_uuid: "ce0da1f507184ce183421d9b93b39382" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce0da1f507184ce183421d9b93b39382" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33763 } } }
I20260924 01:01:12.290848 29406 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce0da1f507184ce183421d9b93b39382 [sys.catalog]: This master's current role is: LEADER
I20260924 01:01:12.292783 29405 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce0da1f507184ce183421d9b93b39382 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce0da1f507184ce183421d9b93b39382" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce0da1f507184ce183421d9b93b39382" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33763 } } }
I20260924 01:01:12.293210 29405 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce0da1f507184ce183421d9b93b39382 [sys.catalog]: This master's current role is: LEADER
I20260924 01:01:12.295825 29411 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 01:01:12.300459 29411 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 01:01:12.309998 29411 catalog_manager.cc:1347] Generated new cluster ID: dd88fc5e7afd4692881716a41dd1bed0
I20260924 01:01:12.310158 29411 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 01:01:12.323858 29411 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 01:01:12.325130 29411 catalog_manager.cc:1504] Loading token signing keys...
I20260924 01:01:12.351521 29411 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce0da1f507184ce183421d9b93b39382: Generated new TSK 0
I20260924 01:01:12.352432 29411 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 01:01:12.357685 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 29327
W20260924 01:01:12.358847 29389 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:59223
W20260924 01:01:12.359586 8450 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 29327: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260924 01:01:12.359732 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 29327
2026-09-24T01:01:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1926 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790211343099518-8450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 01:01:12 dist-test-slave-mmhs krb5kdc[29425](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 01:01:12 dist-test-slave-mmhs krb5kdc[29425](info): set up 2 sockets
Sep 24 01:01:12 dist-test-slave-mmhs krb5kdc[29425](info): commencing operation
krb5kdc: starting...
W20260924 01:01:13.476325 8450 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.004s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Sep 24 01:01:13 dist-test-slave-mmhs krb5kdc[29425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T01:01:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T01:01:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 01:01:13.751591 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43607
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43607
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 01:01:14.084019 29438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:01:14.084324 29438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:01:14.084488 29438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:01:14.091768 29438 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 01:01:14.091890 29438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:01:14.091979 29438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:01:14.092094 29438 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 01:01:14.092181 29438 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 01:01:14.102370 29438 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43607
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:01:14.103701 29438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:01:14.105062 29438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:01:14.116901 29443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:14.116793 29444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:14.116683 29446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:01:14.118010 29438 server_base.cc:1034] running on GCE node
I20260924 01:01:14.118851 29438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:01:14.120329 29438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:01:14.121591 29438 hybrid_clock.cc:648] HybridClock initialized: now 1790211674121529 us; error 67 us; skew 500 ppm
I20260924 01:01:14.121976 29438 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 01:01:14 dist-test-slave-mmhs krb5kdc[29425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211674, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 01:01:14.125041 29438 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 01:01:14.126528 29438 webserver.cc:460] Webserver started at http://127.0.0.1:41769/ using document root <none> and password file <none>
I20260924 01:01:14.127264 29438 fs_manager.cc:359] Metadata directory not provided
I20260924 01:01:14.127408 29438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:01:14.127713 29438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:01:14.130673 29438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "0481fb848a364f7ba0c53e9b9fd79f90"
format_stamp: "Formatted at 2026-09-24 01:01:14 on dist-test-slave-mmhs"
server_key: "33dfcc9c7527f9bdd7345cafdac2bc72"
server_key_iv: "029068e3b46fae93e1ebe8b92379a761"
server_key_version: "encryptionkey@0"
I20260924 01:01:14.131467 29438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "0481fb848a364f7ba0c53e9b9fd79f90"
format_stamp: "Formatted at 2026-09-24 01:01:14 on dist-test-slave-mmhs"
server_key: "33dfcc9c7527f9bdd7345cafdac2bc72"
server_key_iv: "029068e3b46fae93e1ebe8b92379a761"
server_key_version: "encryptionkey@0"
I20260924 01:01:14.136451 29438 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260924 01:01:14.139889 29453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:01:14.141127 29438 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260924 01:01:14.141309 29438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "0481fb848a364f7ba0c53e9b9fd79f90"
format_stamp: "Formatted at 2026-09-24 01:01:14 on dist-test-slave-mmhs"
server_key: "33dfcc9c7527f9bdd7345cafdac2bc72"
server_key_iv: "029068e3b46fae93e1ebe8b92379a761"
server_key_version: "encryptionkey@0"
I20260924 01:01:14.141506 29438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:01:14.147257 29438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:01:14.151064 29438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:01:14.151355 29438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:01:14.170151 29438 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43607
I20260924 01:01:14.170192 29504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43607 every 8 connection(s)
I20260924 01:01:14.171607 29438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 01:01:14.175302 29505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:01:14.177968 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 29438
I20260924 01:01:14.178160 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 01:01:14.178755 8450 external_mini_cluster.cc:1428] Setting key 19f5e6b65f0dd397fd1e7685f0e89658
I20260924 01:01:14.186632 29508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:14.180603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57897 (local address 127.0.0.1:43607)
0924 01:01:14.181437 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:14.181448 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:14.181491 (+ 43us) server_negotiation.cc:407] Connection header received
0924 01:01:14.183203 (+ 1712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:14.183226 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:14.183561 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:14.183884 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:14.185007 (+ 1123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:14.185016 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:14.185028 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:01:14.185043 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:14.185104 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:01:14.185124 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:14.185304 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:14.185442 (+ 138us) server_negotiation.cc:299] Negotiation successful
0924 01:01:14.185662 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":405,"threads_started":1}
I20260924 01:01:14.187436 29505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0481fb848a364f7ba0c53e9b9fd79f90: Bootstrap starting.
I20260924 01:01:14.190231 8450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 01:01:14.190781 29505 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0481fb848a364f7ba0c53e9b9fd79f90: Neither blocks nor log segments found. Creating new log.
I20260924 01:01:14.192013 29505 log.cc:824] T 00000000000000000000000000000000 P 0481fb848a364f7ba0c53e9b9fd79f90: Log is configured to *not* fsync() on all Append() calls
I20260924 01:01:14.195379 29505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0481fb848a364f7ba0c53e9b9fd79f90: No bootstrap required, opened a new log
I20260924 01:01:14.198028 29508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:14.195382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36323 (local address 127.0.0.1:43607)
0924 01:01:14.195562 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:14.195567 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:14.196047 (+ 480us) server_negotiation.cc:407] Connection header received
0924 01:01:14.196196 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:14.196201 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:14.196266 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:14.196376 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:14.197357 (+ 981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:14.197362 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:14.197366 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:01:14.197386 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:14.197432 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:01:14.197460 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:14.197591 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:14.197667 (+ 76us) server_negotiation.cc:299] Negotiation successful
0924 01:01:14.197728 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260924 01:01:14.203382 29505 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0481fb848a364f7ba0c53e9b9fd79f90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0481fb848a364f7ba0c53e9b9fd79f90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43607 } }
I20260924 01:01:14.203783 29505 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0481fb848a364f7ba0c53e9b9fd79f90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 01:01:14.203903 29505 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0481fb848a364f7ba0c53e9b9fd79f90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0481fb848a364f7ba0c53e9b9fd79f90, State: Initialized, Role: FOLLOWER
I20260924 01:01:14.204572 29505 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0481fb848a364f7ba0c53e9b9fd79f90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0481fb848a364f7ba0c53e9b9fd79f90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43607 } }
I20260924 01:01:14.204828 29505 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0481fb848a364f7ba0c53e9b9fd79f90 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 01:01:14.204983 29505 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0481fb848a364f7ba0c53e9b9fd79f90 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 01:01:14.205178 29505 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0481fb848a364f7ba0c53e9b9fd79f90 [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:01:14.207506 29505 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0481fb848a364f7ba0c53e9b9fd79f90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0481fb848a364f7ba0c53e9b9fd79f90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43607 } }
I20260924 01:01:14.208027 29505 leader_election.cc:302] T 00000000000000000000000000000000 P 0481fb848a364f7ba0c53e9b9fd79f90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0481fb848a364f7ba0c53e9b9fd79f90; no voters:
I20260924 01:01:14.208628 29505 leader_election.cc:288] T 00000000000000000000000000000000 P 0481fb848a364f7ba0c53e9b9fd79f90 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 01:01:14.208786 29515 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0481fb848a364f7ba0c53e9b9fd79f90 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 01:01:14.209831 29515 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0481fb848a364f7ba0c53e9b9fd79f90 [term 1 LEADER]: Becoming Leader. State: Replica: 0481fb848a364f7ba0c53e9b9fd79f90, State: Running, Role: LEADER
I20260924 01:01:14.210491 29515 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0481fb848a364f7ba0c53e9b9fd79f90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0481fb848a364f7ba0c53e9b9fd79f90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43607 } }
I20260924 01:01:14.210889 29505 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0481fb848a364f7ba0c53e9b9fd79f90 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 01:01:14.213500 29517 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0481fb848a364f7ba0c53e9b9fd79f90 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0481fb848a364f7ba0c53e9b9fd79f90. Latest consensus state: current_term: 1 leader_uuid: "0481fb848a364f7ba0c53e9b9fd79f90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0481fb848a364f7ba0c53e9b9fd79f90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43607 } } }
I20260924 01:01:14.213898 29517 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0481fb848a364f7ba0c53e9b9fd79f90 [sys.catalog]: This master's current role is: LEADER
I20260924 01:01:14.215194 29516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0481fb848a364f7ba0c53e9b9fd79f90 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0481fb848a364f7ba0c53e9b9fd79f90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0481fb848a364f7ba0c53e9b9fd79f90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43607 } } }
I20260924 01:01:14.215484 29516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0481fb848a364f7ba0c53e9b9fd79f90 [sys.catalog]: This master's current role is: LEADER
I20260924 01:01:14.215588 29523 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 01:01:14.220474 29523 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 01:01:14.229720 29523 catalog_manager.cc:1347] Generated new cluster ID: d422594ac2c74939941db40e51a7fdd2
I20260924 01:01:14.229837 29523 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 01:01:14.252938 29523 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 01:01:14.253429 29523 catalog_manager.cc:1504] Loading token signing keys...
I20260924 01:01:14.267143 29523 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0481fb848a364f7ba0c53e9b9fd79f90: Generated new TSK 0
I20260924 01:01:14.268253 29523 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 01:01:14.274751 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 29438
W20260924 01:01:14.276052 29500 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:57897
W20260924 01:01:14.276813 8450 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 29438: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260924 01:01:14.276969 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 29438
2026-09-24T01:01:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1913 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790211343099518-8450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 01:01:14 dist-test-slave-mmhs krb5kdc[29536](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 01:01:14 dist-test-slave-mmhs krb5kdc[29536](info): set up 2 sockets
Sep 24 01:01:14 dist-test-slave-mmhs krb5kdc[29536](info): commencing operation
krb5kdc: starting...
W20260924 01:01:15.396447 8450 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.008s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Sep 24 01:01:15 dist-test-slave-mmhs krb5kdc[29536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T01:01:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T01:01:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 01:01:15.677724 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44907
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44907
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 01:01:16.007598 29549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:01:16.007942 29549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:01:16.008116 29549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:01:16.015612 29549 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 01:01:16.015765 29549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:01:16.015868 29549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:01:16.015985 29549 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 01:01:16.016088 29549 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 01:01:16.026103 29549 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44907
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:01:16.027421 29549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:01:16.028716 29549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:01:16.040256 29554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:16.040347 29555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:16.040325 29557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:01:16.041549 29549 server_base.cc:1034] running on GCE node
I20260924 01:01:16.042390 29549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:01:16.043754 29549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:01:16.044960 29549 hybrid_clock.cc:648] HybridClock initialized: now 1790211676044921 us; error 40 us; skew 500 ppm
I20260924 01:01:16.045331 29549 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 01:01:16 dist-test-slave-mmhs krb5kdc[29536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211676, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 01:01:16.052433 29549 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 01:01:16.054013 29549 webserver.cc:460] Webserver started at http://127.0.0.1:33917/ using document root <none> and password file <none>
I20260924 01:01:16.054807 29549 fs_manager.cc:359] Metadata directory not provided
I20260924 01:01:16.054955 29549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:01:16.055259 29549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:01:16.058230 29549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "88e7405b0884401da05f13268702c02d"
format_stamp: "Formatted at 2026-09-24 01:01:16 on dist-test-slave-mmhs"
server_key: "959f74d572408b3a1132e1050bde51e8"
server_key_iv: "e43b5e857ac46299b3746f23d951e3e3"
server_key_version: "encryptionkey@0"
I20260924 01:01:16.059067 29549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "88e7405b0884401da05f13268702c02d"
format_stamp: "Formatted at 2026-09-24 01:01:16 on dist-test-slave-mmhs"
server_key: "959f74d572408b3a1132e1050bde51e8"
server_key_iv: "e43b5e857ac46299b3746f23d951e3e3"
server_key_version: "encryptionkey@0"
I20260924 01:01:16.064232 29549 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260924 01:01:16.067701 29564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:01:16.068960 29549 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260924 01:01:16.069162 29549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "88e7405b0884401da05f13268702c02d"
format_stamp: "Formatted at 2026-09-24 01:01:16 on dist-test-slave-mmhs"
server_key: "959f74d572408b3a1132e1050bde51e8"
server_key_iv: "e43b5e857ac46299b3746f23d951e3e3"
server_key_version: "encryptionkey@0"
I20260924 01:01:16.069367 29549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:01:16.098383 29549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:01:16.102953 29549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:01:16.103271 29549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:01:16.121562 29549 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44907
I20260924 01:01:16.121640 29615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44907 every 8 connection(s)
I20260924 01:01:16.123016 29549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 01:01:16.126204 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 29549
I20260924 01:01:16.126443 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 01:01:16.127013 8450 external_mini_cluster.cc:1428] Setting key bfb55eff586aa1103b18cb2f21f47bc2
I20260924 01:01:16.127195 29616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:01:16.142030 29616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88e7405b0884401da05f13268702c02d: Bootstrap starting.
Sep 24 01:01:16 dist-test-slave-mmhs krb5kdc[29536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260924 01:01:16.145816 29616 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88e7405b0884401da05f13268702c02d: Neither blocks nor log segments found. Creating new log.
I20260924 01:01:16.147259 29616 log.cc:824] T 00000000000000000000000000000000 P 88e7405b0884401da05f13268702c02d: Log is configured to *not* fsync() on all Append() calls
I20260924 01:01:16.150910 29616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88e7405b0884401da05f13268702c02d: No bootstrap required, opened a new log
I20260924 01:01:16.153024 29619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:16.128798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:41447 (local address 127.0.0.1:44907)
0924 01:01:16.129571 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:16.129586 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:16.129635 (+ 49us) server_negotiation.cc:407] Connection header received
0924 01:01:16.132090 (+ 2455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:16.132117 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:16.132665 (+ 548us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:16.133100 (+ 435us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:16.134454 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:16.136480 (+ 2026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:16.138401 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:16.139522 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:16.139716 (+ 194us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:16.143944 (+ 4228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:16.143956 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:16.143974 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:01:16.144030 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:16.148067 (+ 4037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:16.149167 (+ 1100us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:16.149176 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:16.149183 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:16.149385 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:16.149849 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:16.149854 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:16.149857 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:16.150588 (+ 731us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:16.150915 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:16.151343 (+ 428us) server_negotiation.cc:299] Negotiation successful
0924 01:01:16.151723 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":311,"threads_started":1}
I20260924 01:01:16.157081 8450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 01:01:16.160034 29616 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88e7405b0884401da05f13268702c02d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e7405b0884401da05f13268702c02d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44907 } }
I20260924 01:01:16.160470 29616 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88e7405b0884401da05f13268702c02d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 01:01:16.160590 29616 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88e7405b0884401da05f13268702c02d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88e7405b0884401da05f13268702c02d, State: Initialized, Role: FOLLOWER
I20260924 01:01:16.161252 29616 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88e7405b0884401da05f13268702c02d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e7405b0884401da05f13268702c02d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44907 } }
I20260924 01:01:16.161531 29616 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88e7405b0884401da05f13268702c02d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 01:01:16.161724 29616 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88e7405b0884401da05f13268702c02d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 01:01:16.161918 29616 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88e7405b0884401da05f13268702c02d [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:01:16.164182 29616 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88e7405b0884401da05f13268702c02d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e7405b0884401da05f13268702c02d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44907 } }
I20260924 01:01:16.164693 29616 leader_election.cc:302] T 00000000000000000000000000000000 P 88e7405b0884401da05f13268702c02d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88e7405b0884401da05f13268702c02d; no voters:
I20260924 01:01:16.165382 29616 leader_election.cc:288] T 00000000000000000000000000000000 P 88e7405b0884401da05f13268702c02d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 01:01:16.165498 29626 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88e7405b0884401da05f13268702c02d [term 1 FOLLOWER]: Leader election won for term 1
I20260924 01:01:16.166685 29626 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88e7405b0884401da05f13268702c02d [term 1 LEADER]: Becoming Leader. State: Replica: 88e7405b0884401da05f13268702c02d, State: Running, Role: LEADER
I20260924 01:01:16.167307 29626 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88e7405b0884401da05f13268702c02d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e7405b0884401da05f13268702c02d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44907 } }
I20260924 01:01:16.167837 29616 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88e7405b0884401da05f13268702c02d [sys.catalog]: configured and running, proceeding with master startup.
I20260924 01:01:16.170219 29627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88e7405b0884401da05f13268702c02d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88e7405b0884401da05f13268702c02d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e7405b0884401da05f13268702c02d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44907 } } }
I20260924 01:01:16.170709 29627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88e7405b0884401da05f13268702c02d [sys.catalog]: This master's current role is: LEADER
I20260924 01:01:16.172956 29628 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88e7405b0884401da05f13268702c02d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88e7405b0884401da05f13268702c02d. Latest consensus state: current_term: 1 leader_uuid: "88e7405b0884401da05f13268702c02d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e7405b0884401da05f13268702c02d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44907 } } }
I20260924 01:01:16.173444 29628 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88e7405b0884401da05f13268702c02d [sys.catalog]: This master's current role is: LEADER
I20260924 01:01:16.174655 29631 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 01:01:16.178984 29619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:16.161516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:38907 (local address 127.0.0.1:44907)
0924 01:01:16.161689 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:16.161696 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:16.162080 (+ 384us) server_negotiation.cc:407] Connection header received
0924 01:01:16.162279 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:16.162284 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:16.162407 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:16.162516 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:16.163903 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:16.164710 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:16.166250 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:16.167026 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:16.167112 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:16.170328 (+ 3216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:16.170335 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:16.170340 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:01:16.170375 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:16.174745 (+ 4370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:16.175516 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:16.175527 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:16.175533 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:16.175703 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:16.176378 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:16.176385 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:16.176390 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:16.176909 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:16.177234 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:16.178359 (+ 1125us) server_negotiation.cc:299] Negotiation successful
0924 01:01:16.178549 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260924 01:01:16.181006 29631 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 01:01:16.191035 29631 catalog_manager.cc:1347] Generated new cluster ID: f3f49513685046d49be17cc2da3aadb3
I20260924 01:01:16.191157 29631 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 01:01:16.203581 29631 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 01:01:16.204839 29631 catalog_manager.cc:1504] Loading token signing keys...
I20260924 01:01:16.211935 29631 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 88e7405b0884401da05f13268702c02d: Generated new TSK 0
I20260924 01:01:16.212847 29631 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 01:01:16.224613 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 29549
I20260924 01:01:16.225647 29611 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 01:01:16.314970 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 29549
W20260924 01:01:16.332628 29534 connection.cc:535] client connection to 127.0.0.1:44907 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44907): Connection reset by peer (error 104)
2026-09-24T01:01:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2041 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790211343099518-8450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 01:01:16 dist-test-slave-mmhs krb5kdc[29648](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 01:01:16 dist-test-slave-mmhs krb5kdc[29648](info): set up 2 sockets
Sep 24 01:01:16 dist-test-slave-mmhs krb5kdc[29648](info): commencing operation
krb5kdc: starting...
W20260924 01:01:17.436448 8450 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.003s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Sep 24 01:01:17 dist-test-slave-mmhs krb5kdc[29648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T01:01:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T01:01:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 01:01:17.711686 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41419
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41419
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 01:01:18.039101 29661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:01:18.039552 29661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:01:18.039732 29661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:01:18.047104 29661 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 01:01:18.047260 29661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:01:18.047379 29661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:01:18.047487 29661 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 01:01:18.047573 29661 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 01:01:18.057503 29661 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41419
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:01:18.058825 29661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:01:18.060156 29661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:01:18.071907 29666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:18.072038 29667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:18.072103 29669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:01:18.073079 29661 server_base.cc:1034] running on GCE node
I20260924 01:01:18.073859 29661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:01:18.075343 29661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:01:18.076570 29661 hybrid_clock.cc:648] HybridClock initialized: now 1790211678076514 us; error 58 us; skew 500 ppm
I20260924 01:01:18.076928 29661 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 01:01:18 dist-test-slave-mmhs krb5kdc[29648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211678, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 01:01:18.080118 29661 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 01:01:18.081568 29661 webserver.cc:460] Webserver started at http://127.0.0.1:41339/ using document root <none> and password file <none>
I20260924 01:01:18.082248 29661 fs_manager.cc:359] Metadata directory not provided
I20260924 01:01:18.082415 29661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:01:18.082710 29661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:01:18.085598 29661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "daec7741f576464e8bf31ab8652fe090"
format_stamp: "Formatted at 2026-09-24 01:01:18 on dist-test-slave-mmhs"
server_key: "3e57c5c220c99ca0d8a9018469460e58"
server_key_iv: "f82c51f64d838416ef75f16079d3a559"
server_key_version: "encryptionkey@0"
I20260924 01:01:18.086417 29661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "daec7741f576464e8bf31ab8652fe090"
format_stamp: "Formatted at 2026-09-24 01:01:18 on dist-test-slave-mmhs"
server_key: "3e57c5c220c99ca0d8a9018469460e58"
server_key_iv: "f82c51f64d838416ef75f16079d3a559"
server_key_version: "encryptionkey@0"
I20260924 01:01:18.091435 29661 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260924 01:01:18.095449 29676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:01:18.096751 29661 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260924 01:01:18.096962 29661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "daec7741f576464e8bf31ab8652fe090"
format_stamp: "Formatted at 2026-09-24 01:01:18 on dist-test-slave-mmhs"
server_key: "3e57c5c220c99ca0d8a9018469460e58"
server_key_iv: "f82c51f64d838416ef75f16079d3a559"
server_key_version: "encryptionkey@0"
I20260924 01:01:18.097167 29661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:01:18.113608 29661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:01:18.117614 29661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:01:18.117895 29661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:01:18.136441 29661 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41419
I20260924 01:01:18.136476 29727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41419 every 8 connection(s)
I20260924 01:01:18.137785 29661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 01:01:18.138610 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 29661
I20260924 01:01:18.138813 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 01:01:18.139272 8450 external_mini_cluster.cc:1428] Setting key 147defe80ae3b68af2832bae436c2472
I20260924 01:01:18.142412 29728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:01:18.152923 29731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:18.141224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:52471 (local address 127.0.0.1:41419)
0924 01:01:18.141964 (+ 740us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:18.141975 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:18.142014 (+ 39us) server_negotiation.cc:407] Connection header received
0924 01:01:18.143793 (+ 1779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:18.143818 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:18.144192 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:18.144513 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:18.145127 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:18.146543 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:18.148434 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:18.149110 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:18.149242 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:18.150585 (+ 1343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:18.150592 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:18.150605 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:01:18.151145 (+ 540us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:18.151232 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:01:18.151255 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:18.151378 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:18.151719 (+ 341us) server_negotiation.cc:299] Negotiation successful
0924 01:01:18.151909 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":322,"threads_started":1}
I20260924 01:01:18.154978 29728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P daec7741f576464e8bf31ab8652fe090: Bootstrap starting.
I20260924 01:01:18.156857 8450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 01:01:18.158512 29728 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P daec7741f576464e8bf31ab8652fe090: Neither blocks nor log segments found. Creating new log.
I20260924 01:01:18.159759 29728 log.cc:824] T 00000000000000000000000000000000 P daec7741f576464e8bf31ab8652fe090: Log is configured to *not* fsync() on all Append() calls
I20260924 01:01:18.162763 29728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P daec7741f576464e8bf31ab8652fe090: No bootstrap required, opened a new log
I20260924 01:01:18.168547 29731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:18.162329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:36629 (local address 127.0.0.1:41419)
0924 01:01:18.162500 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:18.162506 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:18.162958 (+ 452us) server_negotiation.cc:407] Connection header received
0924 01:01:18.163178 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:18.163186 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:18.163248 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:18.163359 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:18.163871 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:18.164655 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:18.166010 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:18.166520 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:18.166660 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:18.167802 (+ 1142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:18.167806 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:18.167810 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:01:18.167827 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:18.167869 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:01:18.167883 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:18.168036 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:18.168203 (+ 167us) server_negotiation.cc:299] Negotiation successful
0924 01:01:18.168257 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260924 01:01:18.170656 29728 raft_consensus.cc:348] T 00000000000000000000000000000000 P daec7741f576464e8bf31ab8652fe090 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daec7741f576464e8bf31ab8652fe090" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41419 } }
I20260924 01:01:18.171041 29728 raft_consensus.cc:374] T 00000000000000000000000000000000 P daec7741f576464e8bf31ab8652fe090 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 01:01:18.171149 29728 raft_consensus.cc:729] T 00000000000000000000000000000000 P daec7741f576464e8bf31ab8652fe090 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: daec7741f576464e8bf31ab8652fe090, State: Initialized, Role: FOLLOWER
I20260924 01:01:18.171774 29728 consensus_queue.cc:260] T 00000000000000000000000000000000 P daec7741f576464e8bf31ab8652fe090 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daec7741f576464e8bf31ab8652fe090" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41419 } }
I20260924 01:01:18.172019 29728 raft_consensus.cc:388] T 00000000000000000000000000000000 P daec7741f576464e8bf31ab8652fe090 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 01:01:18.172160 29728 raft_consensus.cc:482] T 00000000000000000000000000000000 P daec7741f576464e8bf31ab8652fe090 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 01:01:18.172340 29728 raft_consensus.cc:3037] T 00000000000000000000000000000000 P daec7741f576464e8bf31ab8652fe090 [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:01:18.174605 29728 raft_consensus.cc:504] T 00000000000000000000000000000000 P daec7741f576464e8bf31ab8652fe090 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daec7741f576464e8bf31ab8652fe090" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41419 } }
I20260924 01:01:18.175089 29728 leader_election.cc:302] T 00000000000000000000000000000000 P daec7741f576464e8bf31ab8652fe090 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: daec7741f576464e8bf31ab8652fe090; no voters:
I20260924 01:01:18.175750 29728 leader_election.cc:288] T 00000000000000000000000000000000 P daec7741f576464e8bf31ab8652fe090 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 01:01:18.175882 29738 raft_consensus.cc:2781] T 00000000000000000000000000000000 P daec7741f576464e8bf31ab8652fe090 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 01:01:18.176849 29738 raft_consensus.cc:686] T 00000000000000000000000000000000 P daec7741f576464e8bf31ab8652fe090 [term 1 LEADER]: Becoming Leader. State: Replica: daec7741f576464e8bf31ab8652fe090, State: Running, Role: LEADER
I20260924 01:01:18.177346 29738 consensus_queue.cc:237] T 00000000000000000000000000000000 P daec7741f576464e8bf31ab8652fe090 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daec7741f576464e8bf31ab8652fe090" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41419 } }
I20260924 01:01:18.177835 29728 sys_catalog.cc:564] T 00000000000000000000000000000000 P daec7741f576464e8bf31ab8652fe090 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 01:01:18.180452 29739 sys_catalog.cc:455] T 00000000000000000000000000000000 P daec7741f576464e8bf31ab8652fe090 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "daec7741f576464e8bf31ab8652fe090" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daec7741f576464e8bf31ab8652fe090" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41419 } } }
I20260924 01:01:18.180917 29739 sys_catalog.cc:458] T 00000000000000000000000000000000 P daec7741f576464e8bf31ab8652fe090 [sys.catalog]: This master's current role is: LEADER
I20260924 01:01:18.180596 29740 sys_catalog.cc:455] T 00000000000000000000000000000000 P daec7741f576464e8bf31ab8652fe090 [sys.catalog]: SysCatalogTable state changed. Reason: New leader daec7741f576464e8bf31ab8652fe090. Latest consensus state: current_term: 1 leader_uuid: "daec7741f576464e8bf31ab8652fe090" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daec7741f576464e8bf31ab8652fe090" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41419 } } }
I20260924 01:01:18.181044 29740 sys_catalog.cc:458] T 00000000000000000000000000000000 P daec7741f576464e8bf31ab8652fe090 [sys.catalog]: This master's current role is: LEADER
I20260924 01:01:18.182597 29743 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 01:01:18.188318 29743 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 01:01:18.197088 29743 catalog_manager.cc:1347] Generated new cluster ID: 300e85307b0140d99bd0224d61bed20a
I20260924 01:01:18.197223 29743 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 01:01:18.207123 29743 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 01:01:18.208392 29743 catalog_manager.cc:1504] Loading token signing keys...
I20260924 01:01:18.217338 29743 catalog_manager.cc:5868] T 00000000000000000000000000000000 P daec7741f576464e8bf31ab8652fe090: Generated new TSK 0
I20260924 01:01:18.218176 29743 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 01:01:18.228333 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 29661
W20260924 01:01:18.229538 29723 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.143:52471
W20260924 01:01:18.230381 8450 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 29661: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260924 01:01:18.230520 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 29661
W20260924 01:01:18.250002 29646 connection.cc:535] client connection to 127.0.0.1:41419 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41419): Connection reset by peer (error 104)
2026-09-24T01:01:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1917 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790211343099518-8450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 01:01:18 dist-test-slave-mmhs krb5kdc[29759](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 01:01:18 dist-test-slave-mmhs krb5kdc[29759](info): set up 2 sockets
Sep 24 01:01:18 dist-test-slave-mmhs krb5kdc[29759](info): commencing operation
krb5kdc: starting...
W20260924 01:01:19.352389 8450 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Sep 24 01:01:19 dist-test-slave-mmhs krb5kdc[29759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T01:01:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T01:01:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 01:01:19.626029 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33887
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33887
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 01:01:19.956516 29772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:01:19.956785 29772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:01:19.956948 29772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:01:19.964658 29772 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 01:01:19.964761 29772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:01:19.964855 29772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:01:19.964942 29772 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 01:01:19.965030 29772 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 01:01:19.974705 29772 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33887
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:01:19.976275 29772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:01:19.977552 29772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:01:19.994943 29778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:19.994984 29780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:19.995064 29777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:01:19.995754 29772 server_base.cc:1034] running on GCE node
I20260924 01:01:19.996552 29772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:01:19.998004 29772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:01:19.999238 29772 hybrid_clock.cc:648] HybridClock initialized: now 1790211679999200 us; error 52 us; skew 500 ppm
I20260924 01:01:19.999600 29772 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 01:01:20 dist-test-slave-mmhs krb5kdc[29759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211679, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 01:01:20.002936 29772 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 01:01:20.004441 29772 webserver.cc:460] Webserver started at http://127.0.0.1:39717/ using document root <none> and password file <none>
I20260924 01:01:20.005128 29772 fs_manager.cc:359] Metadata directory not provided
I20260924 01:01:20.005275 29772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:01:20.005584 29772 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:01:20.008582 29772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "f003ed9ec47e4eaf8cf3a30c0e37d59a"
format_stamp: "Formatted at 2026-09-24 01:01:20 on dist-test-slave-mmhs"
server_key: "afc4e32b1123cca42aae255e31b24762"
server_key_iv: "335e33f2e37dc751d377d6a91bc6d4b0"
server_key_version: "encryptionkey@0"
I20260924 01:01:20.009348 29772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "f003ed9ec47e4eaf8cf3a30c0e37d59a"
format_stamp: "Formatted at 2026-09-24 01:01:20 on dist-test-slave-mmhs"
server_key: "afc4e32b1123cca42aae255e31b24762"
server_key_iv: "335e33f2e37dc751d377d6a91bc6d4b0"
server_key_version: "encryptionkey@0"
I20260924 01:01:20.014669 29772 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260924 01:01:20.018107 29787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:01:20.019467 29772 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 01:01:20.019666 29772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "f003ed9ec47e4eaf8cf3a30c0e37d59a"
format_stamp: "Formatted at 2026-09-24 01:01:20 on dist-test-slave-mmhs"
server_key: "afc4e32b1123cca42aae255e31b24762"
server_key_iv: "335e33f2e37dc751d377d6a91bc6d4b0"
server_key_version: "encryptionkey@0"
I20260924 01:01:20.019877 29772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:01:20.025784 29772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:01:20.029697 29772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:01:20.029985 29772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:01:20.049381 29772 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33887
I20260924 01:01:20.049441 29838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33887 every 8 connection(s)
I20260924 01:01:20.050854 29772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 01:01:20.051996 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 29772
I20260924 01:01:20.052183 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 01:01:20.052604 8450 external_mini_cluster.cc:1428] Setting key 85eec9013b09e68e00840f741b986d48
I20260924 01:01:20.055356 29839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:01:20.061336 29842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:20.054474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:57581 (local address 127.0.0.1:33887)
0924 01:01:20.055247 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:20.055262 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:20.055321 (+ 59us) server_negotiation.cc:407] Connection header received
0924 01:01:20.057580 (+ 2259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:20.057608 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:20.058021 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:20.058467 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:20.059422 (+ 955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:20.059429 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:20.059445 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:01:20.059461 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:20.059557 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:01:20.059582 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:20.059802 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:20.059969 (+ 167us) server_negotiation.cc:299] Negotiation successful
0924 01:01:20.060161 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":523,"thread_start_us":305,"threads_started":1}
I20260924 01:01:20.064872 8450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 01:01:20.068146 29839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f003ed9ec47e4eaf8cf3a30c0e37d59a: Bootstrap starting.
I20260924 01:01:20.071666 29839 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f003ed9ec47e4eaf8cf3a30c0e37d59a: Neither blocks nor log segments found. Creating new log.
I20260924 01:01:20.071693 29842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:20.069246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:52405 (local address 127.0.0.1:33887)
0924 01:01:20.069414 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:20.069422 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:20.069836 (+ 414us) server_negotiation.cc:407] Connection header received
0924 01:01:20.070017 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:20.070023 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:20.070077 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:20.070177 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:20.071040 (+ 863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:20.071045 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:20.071050 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:01:20.071066 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:20.071115 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:01:20.071129 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:20.071245 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:20.071356 (+ 111us) server_negotiation.cc:299] Negotiation successful
0924 01:01:20.071417 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260924 01:01:20.072805 29839 log.cc:824] T 00000000000000000000000000000000 P f003ed9ec47e4eaf8cf3a30c0e37d59a: Log is configured to *not* fsync() on all Append() calls
I20260924 01:01:20.075392 29839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f003ed9ec47e4eaf8cf3a30c0e37d59a: No bootstrap required, opened a new log
I20260924 01:01:20.082576 29839 raft_consensus.cc:348] T 00000000000000000000000000000000 P f003ed9ec47e4eaf8cf3a30c0e37d59a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f003ed9ec47e4eaf8cf3a30c0e37d59a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33887 } }
I20260924 01:01:20.082877 29839 raft_consensus.cc:374] T 00000000000000000000000000000000 P f003ed9ec47e4eaf8cf3a30c0e37d59a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 01:01:20.082948 29839 raft_consensus.cc:729] T 00000000000000000000000000000000 P f003ed9ec47e4eaf8cf3a30c0e37d59a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003ed9ec47e4eaf8cf3a30c0e37d59a, State: Initialized, Role: FOLLOWER
I20260924 01:01:20.083460 29839 consensus_queue.cc:260] T 00000000000000000000000000000000 P f003ed9ec47e4eaf8cf3a30c0e37d59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f003ed9ec47e4eaf8cf3a30c0e37d59a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33887 } }
I20260924 01:01:20.083674 29839 raft_consensus.cc:388] T 00000000000000000000000000000000 P f003ed9ec47e4eaf8cf3a30c0e37d59a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 01:01:20.083827 29839 raft_consensus.cc:482] T 00000000000000000000000000000000 P f003ed9ec47e4eaf8cf3a30c0e37d59a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 01:01:20.084018 29839 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f003ed9ec47e4eaf8cf3a30c0e37d59a [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:01:20.086046 29839 raft_consensus.cc:504] T 00000000000000000000000000000000 P f003ed9ec47e4eaf8cf3a30c0e37d59a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f003ed9ec47e4eaf8cf3a30c0e37d59a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33887 } }
I20260924 01:01:20.086572 29839 leader_election.cc:302] T 00000000000000000000000000000000 P f003ed9ec47e4eaf8cf3a30c0e37d59a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f003ed9ec47e4eaf8cf3a30c0e37d59a; no voters:
I20260924 01:01:20.087106 29839 leader_election.cc:288] T 00000000000000000000000000000000 P f003ed9ec47e4eaf8cf3a30c0e37d59a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 01:01:20.087244 29849 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f003ed9ec47e4eaf8cf3a30c0e37d59a [term 1 FOLLOWER]: Leader election won for term 1
I20260924 01:01:20.087940 29849 raft_consensus.cc:686] T 00000000000000000000000000000000 P f003ed9ec47e4eaf8cf3a30c0e37d59a [term 1 LEADER]: Becoming Leader. State: Replica: f003ed9ec47e4eaf8cf3a30c0e37d59a, State: Running, Role: LEADER
I20260924 01:01:20.088449 29849 consensus_queue.cc:237] T 00000000000000000000000000000000 P f003ed9ec47e4eaf8cf3a30c0e37d59a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f003ed9ec47e4eaf8cf3a30c0e37d59a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33887 } }
I20260924 01:01:20.089010 29839 sys_catalog.cc:564] T 00000000000000000000000000000000 P f003ed9ec47e4eaf8cf3a30c0e37d59a [sys.catalog]: configured and running, proceeding with master startup.
I20260924 01:01:20.091351 29850 sys_catalog.cc:455] T 00000000000000000000000000000000 P f003ed9ec47e4eaf8cf3a30c0e37d59a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f003ed9ec47e4eaf8cf3a30c0e37d59a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f003ed9ec47e4eaf8cf3a30c0e37d59a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33887 } } }
I20260924 01:01:20.091799 29850 sys_catalog.cc:458] T 00000000000000000000000000000000 P f003ed9ec47e4eaf8cf3a30c0e37d59a [sys.catalog]: This master's current role is: LEADER
I20260924 01:01:20.092353 29851 sys_catalog.cc:455] T 00000000000000000000000000000000 P f003ed9ec47e4eaf8cf3a30c0e37d59a [sys.catalog]: SysCatalogTable state changed. Reason: New leader f003ed9ec47e4eaf8cf3a30c0e37d59a. Latest consensus state: current_term: 1 leader_uuid: "f003ed9ec47e4eaf8cf3a30c0e37d59a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f003ed9ec47e4eaf8cf3a30c0e37d59a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33887 } } }
I20260924 01:01:20.092690 29851 sys_catalog.cc:458] T 00000000000000000000000000000000 P f003ed9ec47e4eaf8cf3a30c0e37d59a [sys.catalog]: This master's current role is: LEADER
I20260924 01:01:20.094900 29857 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 01:01:20.100658 29857 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 01:01:20.109961 29857 catalog_manager.cc:1347] Generated new cluster ID: a1d2c79194744255acf08b16bf5f3c37
I20260924 01:01:20.110072 29857 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 01:01:20.132408 29857 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 01:01:20.132962 29857 catalog_manager.cc:1504] Loading token signing keys...
I20260924 01:01:20.148059 29857 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f003ed9ec47e4eaf8cf3a30c0e37d59a: Generated new TSK 0
I20260924 01:01:20.149166 29857 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 01:01:20.158360 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 29772
W20260924 01:01:20.159580 29834 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.143:57581
W20260924 01:01:20.160298 8450 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 29772: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260924 01:01:20.160457 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 29772
2026-09-24T01:01:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1927 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790211343099518-8450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 01:01:20 dist-test-slave-mmhs krb5kdc[29870](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 01:01:20 dist-test-slave-mmhs krb5kdc[29870](info): set up 2 sockets
Sep 24 01:01:20 dist-test-slave-mmhs krb5kdc[29870](info): commencing operation
krb5kdc: starting...
W20260924 01:01:21.296464 8450 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Sep 24 01:01:21 dist-test-slave-mmhs krb5kdc[29870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T01:01:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T01:01:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 01:01:21.569459 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40525
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40525
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 01:01:21.901110 29883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:01:21.901477 29883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:01:21.901654 29883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:01:21.909164 29883 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 01:01:21.909318 29883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:01:21.909433 29883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:01:21.909540 29883 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 01:01:21.909615 29883 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 01:01:21.919596 29883 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40525
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:01:21.920918 29883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:01:21.922216 29883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:01:21.934052 29888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:21.934091 29889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:21.934127 29891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:01:21.934918 29883 server_base.cc:1034] running on GCE node
I20260924 01:01:21.935648 29883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:01:21.937115 29883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:01:21.938380 29883 hybrid_clock.cc:648] HybridClock initialized: now 1790211681938335 us; error 60 us; skew 500 ppm
I20260924 01:01:21.938730 29883 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 01:01:21 dist-test-slave-mmhs krb5kdc[29870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211681, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 01:01:21.952193 29883 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 01:01:21.953771 29883 webserver.cc:460] Webserver started at http://127.0.0.1:40569/ using document root <none> and password file <none>
I20260924 01:01:21.954547 29883 fs_manager.cc:359] Metadata directory not provided
I20260924 01:01:21.954661 29883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:01:21.954922 29883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:01:21.957890 29883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "23e9c019f9d743f8a6a90356da82ee85"
format_stamp: "Formatted at 2026-09-24 01:01:21 on dist-test-slave-mmhs"
server_key: "2c2b8072c3d703a42e6b179b3bcaa742"
server_key_iv: "a511a1bc36024daf95a00883a2bc44f7"
server_key_version: "encryptionkey@0"
I20260924 01:01:21.958670 29883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "23e9c019f9d743f8a6a90356da82ee85"
format_stamp: "Formatted at 2026-09-24 01:01:21 on dist-test-slave-mmhs"
server_key: "2c2b8072c3d703a42e6b179b3bcaa742"
server_key_iv: "a511a1bc36024daf95a00883a2bc44f7"
server_key_version: "encryptionkey@0"
I20260924 01:01:21.963726 29883 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260924 01:01:21.967344 29898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:01:21.968894 29883 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260924 01:01:21.969105 29883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "23e9c019f9d743f8a6a90356da82ee85"
format_stamp: "Formatted at 2026-09-24 01:01:21 on dist-test-slave-mmhs"
server_key: "2c2b8072c3d703a42e6b179b3bcaa742"
server_key_iv: "a511a1bc36024daf95a00883a2bc44f7"
server_key_version: "encryptionkey@0"
I20260924 01:01:21.969336 29883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:01:22.004148 29883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:01:22.008280 29883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:01:22.008589 29883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:01:22.027987 29883 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40525
I20260924 01:01:22.028036 29949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40525 every 8 connection(s)
I20260924 01:01:22.029352 29883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 01:01:22.033082 29950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:01:22.037782 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 29883
I20260924 01:01:22.037955 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 01:01:22.038451 8450 external_mini_cluster.cc:1428] Setting key 0601aa58e9fd298e04413db111e08d68
I20260924 01:01:22.044248 29950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23e9c019f9d743f8a6a90356da82ee85: Bootstrap starting.
I20260924 01:01:22.047791 29950 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23e9c019f9d743f8a6a90356da82ee85: Neither blocks nor log segments found. Creating new log.
I20260924 01:01:22.048923 29950 log.cc:824] T 00000000000000000000000000000000 P 23e9c019f9d743f8a6a90356da82ee85: Log is configured to *not* fsync() on all Append() calls
I20260924 01:01:22.052186 29950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23e9c019f9d743f8a6a90356da82ee85: No bootstrap required, opened a new log
Sep 24 01:01:22 dist-test-slave-mmhs krb5kdc[29870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260924 01:01:22.059908 29950 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23e9c019f9d743f8a6a90356da82ee85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23e9c019f9d743f8a6a90356da82ee85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40525 } }
I20260924 01:01:22.060305 29950 raft_consensus.cc:374] T 00000000000000000000000000000000 P 23e9c019f9d743f8a6a90356da82ee85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 01:01:22.060424 29950 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23e9c019f9d743f8a6a90356da82ee85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23e9c019f9d743f8a6a90356da82ee85, State: Initialized, Role: FOLLOWER
I20260924 01:01:22.061033 29950 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23e9c019f9d743f8a6a90356da82ee85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23e9c019f9d743f8a6a90356da82ee85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40525 } }
I20260924 01:01:22.061311 29950 raft_consensus.cc:388] T 00000000000000000000000000000000 P 23e9c019f9d743f8a6a90356da82ee85 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 01:01:22.061467 29950 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23e9c019f9d743f8a6a90356da82ee85 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 01:01:22.061650 29950 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23e9c019f9d743f8a6a90356da82ee85 [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:01:22.061969 29953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:22.040101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:48857 (local address 127.0.0.1:40525)
0924 01:01:22.040879 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:22.040893 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:22.040933 (+ 40us) server_negotiation.cc:407] Connection header received
0924 01:01:22.042742 (+ 1809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:22.042764 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:22.043138 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:22.043445 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:22.045178 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:22.046557 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:22.048388 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:22.049045 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:22.049226 (+ 181us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:22.053942 (+ 4716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:22.053950 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:22.053964 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:01:22.054009 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:22.057233 (+ 3224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:22.058043 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:22.058050 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:22.058055 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:22.058232 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:22.058762 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:22.058766 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:22.058768 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:22.059318 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:22.059656 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:22.060155 (+ 499us) server_negotiation.cc:299] Negotiation successful
0924 01:01:22.060741 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":391,"threads_started":1}
I20260924 01:01:22.064175 29950 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23e9c019f9d743f8a6a90356da82ee85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23e9c019f9d743f8a6a90356da82ee85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40525 } }
I20260924 01:01:22.064697 29950 leader_election.cc:302] T 00000000000000000000000000000000 P 23e9c019f9d743f8a6a90356da82ee85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 23e9c019f9d743f8a6a90356da82ee85; no voters:
I20260924 01:01:22.065397 29950 leader_election.cc:288] T 00000000000000000000000000000000 P 23e9c019f9d743f8a6a90356da82ee85 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 01:01:22.065523 29955 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 23e9c019f9d743f8a6a90356da82ee85 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 01:01:22.065852 8450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 01:01:22.067903 29950 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23e9c019f9d743f8a6a90356da82ee85 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 01:01:22.070111 29955 raft_consensus.cc:686] T 00000000000000000000000000000000 P 23e9c019f9d743f8a6a90356da82ee85 [term 1 LEADER]: Becoming Leader. State: Replica: 23e9c019f9d743f8a6a90356da82ee85, State: Running, Role: LEADER
I20260924 01:01:22.070945 29955 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23e9c019f9d743f8a6a90356da82ee85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23e9c019f9d743f8a6a90356da82ee85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40525 } }
I20260924 01:01:22.074084 29957 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23e9c019f9d743f8a6a90356da82ee85 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23e9c019f9d743f8a6a90356da82ee85. Latest consensus state: current_term: 1 leader_uuid: "23e9c019f9d743f8a6a90356da82ee85" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23e9c019f9d743f8a6a90356da82ee85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40525 } } }
I20260924 01:01:22.074644 29957 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23e9c019f9d743f8a6a90356da82ee85 [sys.catalog]: This master's current role is: LEADER
I20260924 01:01:22.074528 29956 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23e9c019f9d743f8a6a90356da82ee85 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "23e9c019f9d743f8a6a90356da82ee85" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23e9c019f9d743f8a6a90356da82ee85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40525 } } }
I20260924 01:01:22.074808 29956 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23e9c019f9d743f8a6a90356da82ee85 [sys.catalog]: This master's current role is: LEADER
I20260924 01:01:22.075569 29965 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 01:01:22.096222 29965 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 01:01:22.101763 29953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:22.071055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:41499 (local address 127.0.0.1:40525)
0924 01:01:22.071253 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:22.071260 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:22.073754 (+ 2494us) server_negotiation.cc:407] Connection header received
0924 01:01:22.074062 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:22.074075 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:22.074172 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:22.074332 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:22.076749 (+ 2417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:22.078339 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:22.080794 (+ 2455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:22.081669 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:22.081799 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:22.086011 (+ 4212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:22.086022 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:22.086030 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:01:22.086079 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:22.097708 (+ 11629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:22.098840 (+ 1132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:22.098845 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:22.098849 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:22.099003 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:22.099663 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:22.099667 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:22.099672 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:22.100105 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:22.101233 (+ 1128us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:22.101369 (+ 136us) server_negotiation.cc:299] Negotiation successful
0924 01:01:22.101511 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260924 01:01:22.107597 29965 catalog_manager.cc:1347] Generated new cluster ID: 451edd6002eb42518e40f1f2dc0b3cfd
I20260924 01:01:22.107720 29965 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 01:01:22.138837 29965 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 01:01:22.140743 29965 catalog_manager.cc:1504] Loading token signing keys...
I20260924 01:01:22.148296 29965 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 23e9c019f9d743f8a6a90356da82ee85: Generated new TSK 0
I20260924 01:01:22.149298 29965 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 01:01:22.159157 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 29883
I20260924 01:01:22.160943 29945 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 01:01:22.251108 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 29883
W20260924 01:01:22.268684 29868 connection.cc:535] client connection to 127.0.0.1:40525 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40525): Connection reset by peer (error 104)
2026-09-24T01:01:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2091 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790211343099518-8450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 01:01:22 dist-test-slave-mmhs krb5kdc[29982](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 01:01:22 dist-test-slave-mmhs krb5kdc[29982](info): set up 2 sockets
Sep 24 01:01:22 dist-test-slave-mmhs krb5kdc[29982](info): commencing operation
krb5kdc: starting...
W20260924 01:01:23.372546 8450 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.001s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Sep 24 01:01:23 dist-test-slave-mmhs krb5kdc[29982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T01:01:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T01:01:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 01:01:23.650707 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44807
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44807
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 01:01:23.985134 29995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:01:23.985477 29995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:01:23.985657 29995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:01:23.992857 29995 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 01:01:23.993018 29995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:01:23.993135 29995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:01:23.993247 29995 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 01:01:23.993331 29995 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 01:01:24.003301 29995 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44807
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:01:24.004668 29995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:01:24.006006 29995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:01:24.018143 30001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:24.018237 30000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:24.018261 30003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:01:24.019187 29995 server_base.cc:1034] running on GCE node
I20260924 01:01:24.020009 29995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:01:24.021461 29995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:01:24.022688 29995 hybrid_clock.cc:648] HybridClock initialized: now 1790211684022646 us; error 37 us; skew 500 ppm
I20260924 01:01:24.023042 29995 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 01:01:24 dist-test-slave-mmhs krb5kdc[29982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211684, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 01:01:24.026075 29995 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 01:01:24.027657 29995 webserver.cc:460] Webserver started at http://127.0.0.1:41671/ using document root <none> and password file <none>
I20260924 01:01:24.028326 29995 fs_manager.cc:359] Metadata directory not provided
I20260924 01:01:24.028473 29995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:01:24.028781 29995 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:01:24.031785 29995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "545b3117de414f0c93f473ab7a540873"
format_stamp: "Formatted at 2026-09-24 01:01:24 on dist-test-slave-mmhs"
server_key: "655d3fb6023481f3fad3f56cfdbef641"
server_key_iv: "d188d00dddca25ba2725c4e61935f896"
server_key_version: "encryptionkey@0"
I20260924 01:01:24.032552 29995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "545b3117de414f0c93f473ab7a540873"
format_stamp: "Formatted at 2026-09-24 01:01:24 on dist-test-slave-mmhs"
server_key: "655d3fb6023481f3fad3f56cfdbef641"
server_key_iv: "d188d00dddca25ba2725c4e61935f896"
server_key_version: "encryptionkey@0"
I20260924 01:01:24.037655 29995 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260924 01:01:24.041220 30010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:01:24.042764 29995 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 01:01:24.042951 29995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "545b3117de414f0c93f473ab7a540873"
format_stamp: "Formatted at 2026-09-24 01:01:24 on dist-test-slave-mmhs"
server_key: "655d3fb6023481f3fad3f56cfdbef641"
server_key_iv: "d188d00dddca25ba2725c4e61935f896"
server_key_version: "encryptionkey@0"
I20260924 01:01:24.043159 29995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:01:24.059077 29995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:01:24.062996 29995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:01:24.063287 29995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:01:24.082134 29995 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44807
I20260924 01:01:24.082165 30061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44807 every 8 connection(s)
I20260924 01:01:24.083571 29995 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 01:01:24.086460 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 29995
I20260924 01:01:24.086632 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 01:01:24.087101 8450 external_mini_cluster.cc:1428] Setting key 4f77159c281eabd9d0f9df46d794dc6b
I20260924 01:01:24.087543 30062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:01:24.102566 30065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:24.088912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:36163 (local address 127.0.0.1:44807)
0924 01:01:24.089736 (+ 824us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:24.089751 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:24.089801 (+ 50us) server_negotiation.cc:407] Connection header received
0924 01:01:24.092324 (+ 2523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:24.092353 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:24.092906 (+ 553us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:24.093350 (+ 444us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:24.093879 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:24.095892 (+ 2013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:24.097288 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:24.098395 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:24.098582 (+ 187us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:24.099752 (+ 1170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:24.099762 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:24.099785 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:01:24.100532 (+ 747us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:24.100635 (+ 103us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:01:24.100665 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:24.100802 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:24.101074 (+ 272us) server_negotiation.cc:299] Negotiation successful
0924 01:01:24.101251 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":317,"threads_started":1}
I20260924 01:01:24.102703 30062 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 545b3117de414f0c93f473ab7a540873: Bootstrap starting.
I20260924 01:01:24.105980 30062 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 545b3117de414f0c93f473ab7a540873: Neither blocks nor log segments found. Creating new log.
I20260924 01:01:24.106417 8450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 01:01:24.107151 30062 log.cc:824] T 00000000000000000000000000000000 P 545b3117de414f0c93f473ab7a540873: Log is configured to *not* fsync() on all Append() calls
I20260924 01:01:24.110143 30062 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 545b3117de414f0c93f473ab7a540873: No bootstrap required, opened a new log
I20260924 01:01:24.117398 30065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:24.111148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:52339 (local address 127.0.0.1:44807)
0924 01:01:24.111344 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:24.111352 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:24.111717 (+ 365us) server_negotiation.cc:407] Connection header received
0924 01:01:24.111936 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:24.111940 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:24.112010 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:24.112116 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:24.112676 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:24.113469 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:24.114833 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:24.115298 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:24.115457 (+ 159us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:24.116635 (+ 1178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:24.116639 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:24.116643 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:01:24.116660 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:24.116697 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:01:24.116714 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:24.116850 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:24.117052 (+ 202us) server_negotiation.cc:299] Negotiation successful
0924 01:01:24.117100 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260924 01:01:24.118599 30062 raft_consensus.cc:348] T 00000000000000000000000000000000 P 545b3117de414f0c93f473ab7a540873 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545b3117de414f0c93f473ab7a540873" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44807 } }
I20260924 01:01:24.119083 30062 raft_consensus.cc:374] T 00000000000000000000000000000000 P 545b3117de414f0c93f473ab7a540873 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 01:01:24.119194 30062 raft_consensus.cc:729] T 00000000000000000000000000000000 P 545b3117de414f0c93f473ab7a540873 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 545b3117de414f0c93f473ab7a540873, State: Initialized, Role: FOLLOWER
I20260924 01:01:24.119797 30062 consensus_queue.cc:260] T 00000000000000000000000000000000 P 545b3117de414f0c93f473ab7a540873 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545b3117de414f0c93f473ab7a540873" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44807 } }
I20260924 01:01:24.120033 30062 raft_consensus.cc:388] T 00000000000000000000000000000000 P 545b3117de414f0c93f473ab7a540873 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 01:01:24.120182 30062 raft_consensus.cc:482] T 00000000000000000000000000000000 P 545b3117de414f0c93f473ab7a540873 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 01:01:24.120369 30062 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 545b3117de414f0c93f473ab7a540873 [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:01:24.122575 30062 raft_consensus.cc:504] T 00000000000000000000000000000000 P 545b3117de414f0c93f473ab7a540873 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545b3117de414f0c93f473ab7a540873" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44807 } }
I20260924 01:01:24.123116 30062 leader_election.cc:302] T 00000000000000000000000000000000 P 545b3117de414f0c93f473ab7a540873 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 545b3117de414f0c93f473ab7a540873; no voters:
I20260924 01:01:24.123689 30062 leader_election.cc:288] T 00000000000000000000000000000000 P 545b3117de414f0c93f473ab7a540873 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 01:01:24.123821 30072 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 545b3117de414f0c93f473ab7a540873 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 01:01:24.125643 30072 raft_consensus.cc:686] T 00000000000000000000000000000000 P 545b3117de414f0c93f473ab7a540873 [term 1 LEADER]: Becoming Leader. State: Replica: 545b3117de414f0c93f473ab7a540873, State: Running, Role: LEADER
I20260924 01:01:24.126156 30072 consensus_queue.cc:237] T 00000000000000000000000000000000 P 545b3117de414f0c93f473ab7a540873 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545b3117de414f0c93f473ab7a540873" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44807 } }
I20260924 01:01:24.126274 30062 sys_catalog.cc:564] T 00000000000000000000000000000000 P 545b3117de414f0c93f473ab7a540873 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 01:01:24.128959 30073 sys_catalog.cc:455] T 00000000000000000000000000000000 P 545b3117de414f0c93f473ab7a540873 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "545b3117de414f0c93f473ab7a540873" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545b3117de414f0c93f473ab7a540873" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44807 } } }
I20260924 01:01:24.129388 30073 sys_catalog.cc:458] T 00000000000000000000000000000000 P 545b3117de414f0c93f473ab7a540873 [sys.catalog]: This master's current role is: LEADER
I20260924 01:01:24.129868 30074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 545b3117de414f0c93f473ab7a540873 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 545b3117de414f0c93f473ab7a540873. Latest consensus state: current_term: 1 leader_uuid: "545b3117de414f0c93f473ab7a540873" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545b3117de414f0c93f473ab7a540873" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44807 } } }
I20260924 01:01:24.130169 30074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 545b3117de414f0c93f473ab7a540873 [sys.catalog]: This master's current role is: LEADER
I20260924 01:01:24.134400 30081 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 01:01:24.139310 30081 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 01:01:24.148038 30081 catalog_manager.cc:1347] Generated new cluster ID: 5751093a7cb44314b1289cc448997c29
I20260924 01:01:24.148149 30081 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 01:01:24.167557 30081 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 01:01:24.169370 30081 catalog_manager.cc:1504] Loading token signing keys...
I20260924 01:01:24.178203 30081 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 545b3117de414f0c93f473ab7a540873: Generated new TSK 0
I20260924 01:01:24.179044 30081 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 01:01:24.193656 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 29995
W20260924 01:01:24.194814 30057 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.143:36163
W20260924 01:01:24.195561 8450 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 29995: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260924 01:01:24.195708 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 29995
W20260924 01:01:24.213229 29980 connection.cc:535] client connection to 127.0.0.1:44807 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44807): Connection reset by peer (error 104)
2026-09-24T01:01:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1948 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790211343099518-8450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 01:01:24 dist-test-slave-mmhs krb5kdc[30093](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 01:01:24 dist-test-slave-mmhs krb5kdc[30093](info): set up 2 sockets
Sep 24 01:01:24 dist-test-slave-mmhs krb5kdc[30093](info): commencing operation
krb5kdc: starting...
W20260924 01:01:25.312384 8450 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Sep 24 01:01:25 dist-test-slave-mmhs krb5kdc[30093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211685, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T01:01:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T01:01:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 01:01:25.587549 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38195
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38195
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 01:01:25.922307 30106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:01:25.922639 30106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:01:25.922735 30106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:01:25.929993 30106 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 01:01:25.930130 30106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:01:25.930186 30106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:01:25.930251 30106 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 01:01:25.930313 30106 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 01:01:25.940307 30106 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38195
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790211343099518-8450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:01:25.941553 30106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:01:25.942857 30106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:01:25.954931 30111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:25.954977 30112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:25.954947 30114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:01:25.956223 30106 server_base.cc:1034] running on GCE node
I20260924 01:01:25.956979 30106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:01:25.958593 30106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:01:25.959836 30106 hybrid_clock.cc:648] HybridClock initialized: now 1790211685959798 us; error 54 us; skew 500 ppm
I20260924 01:01:25.960198 30106 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 01:01:25 dist-test-slave-mmhs krb5kdc[30093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211685, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 01:01:25.963429 30106 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 01:01:25.965093 30106 webserver.cc:460] Webserver started at http://127.0.0.1:32993/ using document root <none> and password file <none>
I20260924 01:01:25.965778 30106 fs_manager.cc:359] Metadata directory not provided
I20260924 01:01:25.965932 30106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:01:25.966243 30106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:01:25.969225 30106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "7984795282f845868fe2eb1bdab86515"
format_stamp: "Formatted at 2026-09-24 01:01:25 on dist-test-slave-mmhs"
server_key: "9f0af17bb29d4b0a8355ab149cbdbe53"
server_key_iv: "3667cafa07ba5fbe36a22347a27dc8a9"
server_key_version: "encryptionkey@0"
I20260924 01:01:25.969997 30106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "7984795282f845868fe2eb1bdab86515"
format_stamp: "Formatted at 2026-09-24 01:01:25 on dist-test-slave-mmhs"
server_key: "9f0af17bb29d4b0a8355ab149cbdbe53"
server_key_iv: "3667cafa07ba5fbe36a22347a27dc8a9"
server_key_version: "encryptionkey@0"
I20260924 01:01:25.975237 30106 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260924 01:01:25.978731 30121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:01:25.980129 30106 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260924 01:01:25.980326 30106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "7984795282f845868fe2eb1bdab86515"
format_stamp: "Formatted at 2026-09-24 01:01:25 on dist-test-slave-mmhs"
server_key: "9f0af17bb29d4b0a8355ab149cbdbe53"
server_key_iv: "3667cafa07ba5fbe36a22347a27dc8a9"
server_key_version: "encryptionkey@0"
I20260924 01:01:25.980532 30106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:01:25.986279 30106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:01:25.990135 30106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:01:25.990509 30106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:01:26.009581 30106 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38195
I20260924 01:01:26.009627 30172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38195 every 8 connection(s)
I20260924 01:01:26.011006 30106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 01:01:26.014698 30173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:01:26.015120 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 30106
I20260924 01:01:26.015304 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 01:01:26.015745 8450 external_mini_cluster.cc:1428] Setting key b520db5198b76120a97f813eb6979479
I20260924 01:01:26.025177 30176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:26.017616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:53347 (local address 127.0.0.1:38195)
0924 01:01:26.018435 (+ 819us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:26.018450 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:26.018510 (+ 60us) server_negotiation.cc:407] Connection header received
0924 01:01:26.021366 (+ 2856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:26.021395 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:26.021848 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:26.022318 (+ 470us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:26.023286 (+ 968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:26.023294 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:26.023310 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:01:26.023326 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:26.023407 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:01:26.023437 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:26.023640 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:26.024057 (+ 417us) server_negotiation.cc:299] Negotiation successful
0924 01:01:26.024298 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":317,"threads_started":1}
I20260924 01:01:26.028709 30173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7984795282f845868fe2eb1bdab86515: Bootstrap starting.
I20260924 01:01:26.028883 8450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 01:01:26.031854 30173 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7984795282f845868fe2eb1bdab86515: Neither blocks nor log segments found. Creating new log.
I20260924 01:01:26.032964 30173 log.cc:824] T 00000000000000000000000000000000 P 7984795282f845868fe2eb1bdab86515: Log is configured to *not* fsync() on all Append() calls
I20260924 01:01:26.035964 30176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:26.033181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:35035 (local address 127.0.0.1:38195)
0924 01:01:26.033366 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:26.033375 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:26.033880 (+ 505us) server_negotiation.cc:407] Connection header received
0924 01:01:26.034052 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:26.034056 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:26.034104 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:26.034192 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:26.035177 (+ 985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:26.035183 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:26.035187 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:01:26.035205 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:26.035249 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:01:26.035267 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:26.035446 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:26.035599 (+ 153us) server_negotiation.cc:299] Negotiation successful
0924 01:01:26.035669 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260924 01:01:26.036494 30173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7984795282f845868fe2eb1bdab86515: No bootstrap required, opened a new log
I20260924 01:01:26.043718 30173 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7984795282f845868fe2eb1bdab86515 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7984795282f845868fe2eb1bdab86515" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38195 } }
I20260924 01:01:26.044082 30173 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7984795282f845868fe2eb1bdab86515 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 01:01:26.044203 30173 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7984795282f845868fe2eb1bdab86515 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7984795282f845868fe2eb1bdab86515, State: Initialized, Role: FOLLOWER
I20260924 01:01:26.044747 30173 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7984795282f845868fe2eb1bdab86515 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7984795282f845868fe2eb1bdab86515" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38195 } }
I20260924 01:01:26.044986 30173 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7984795282f845868fe2eb1bdab86515 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 01:01:26.045153 30173 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7984795282f845868fe2eb1bdab86515 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 01:01:26.045312 30173 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7984795282f845868fe2eb1bdab86515 [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:01:26.047763 30173 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7984795282f845868fe2eb1bdab86515 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7984795282f845868fe2eb1bdab86515" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38195 } }
I20260924 01:01:26.048260 30173 leader_election.cc:302] T 00000000000000000000000000000000 P 7984795282f845868fe2eb1bdab86515 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7984795282f845868fe2eb1bdab86515; no voters:
I20260924 01:01:26.049014 30173 leader_election.cc:288] T 00000000000000000000000000000000 P 7984795282f845868fe2eb1bdab86515 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 01:01:26.049209 30183 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7984795282f845868fe2eb1bdab86515 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 01:01:26.050001 30183 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7984795282f845868fe2eb1bdab86515 [term 1 LEADER]: Becoming Leader. State: Replica: 7984795282f845868fe2eb1bdab86515, State: Running, Role: LEADER
I20260924 01:01:26.050608 30183 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7984795282f845868fe2eb1bdab86515 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7984795282f845868fe2eb1bdab86515" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38195 } }
I20260924 01:01:26.051407 30173 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7984795282f845868fe2eb1bdab86515 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 01:01:26.053563 30185 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7984795282f845868fe2eb1bdab86515 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7984795282f845868fe2eb1bdab86515. Latest consensus state: current_term: 1 leader_uuid: "7984795282f845868fe2eb1bdab86515" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7984795282f845868fe2eb1bdab86515" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38195 } } }
I20260924 01:01:26.053970 30185 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7984795282f845868fe2eb1bdab86515 [sys.catalog]: This master's current role is: LEADER
I20260924 01:01:26.054447 30184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7984795282f845868fe2eb1bdab86515 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7984795282f845868fe2eb1bdab86515" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7984795282f845868fe2eb1bdab86515" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38195 } } }
I20260924 01:01:26.054742 30184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7984795282f845868fe2eb1bdab86515 [sys.catalog]: This master's current role is: LEADER
I20260924 01:01:26.055684 30189 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 01:01:26.063009 30189 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 01:01:26.072082 30189 catalog_manager.cc:1347] Generated new cluster ID: bc41bb4c5d154308a36f9ed5a6016985
I20260924 01:01:26.072213 30189 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 01:01:26.089562 30189 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 01:01:26.089983 30189 catalog_manager.cc:1504] Loading token signing keys...
I20260924 01:01:26.100018 30189 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7984795282f845868fe2eb1bdab86515: Generated new TSK 0
I20260924 01:01:26.100845 30189 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 01:01:26.114507 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 30106
W20260924 01:01:26.115562 30168 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.143:53347
W20260924 01:01:26.116240 8450 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 30106: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260924 01:01:26.116428 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 30106
2026-09-24T01:01:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1916 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790211343099518-8450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 01:01:26 dist-test-slave-mmhs krb5kdc[30204](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 01:01:26 dist-test-slave-mmhs krb5kdc[30204](info): set up 2 sockets
Sep 24 01:01:26 dist-test-slave-mmhs krb5kdc[30204](info): commencing operation
krb5kdc: starting...
W20260924 01:01:27.244415 8450 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.000s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Sep 24 01:01:27 dist-test-slave-mmhs krb5kdc[30204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T01:01:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T01:01:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.64.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
I20260924 01:01:27.530063 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:37695
--webserver_interface=127.8.64.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:33205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:37695
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 01:01:27.861876 30217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:01:27.862166 30217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:01:27.862352 30217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:01:27.869539 30217 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 01:01:27.869652 30217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:01:27.869745 30217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:01:27.869832 30217 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 01:01:27.869900 30217 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 01:01:27.879748 30217 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:33205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:37695
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:37695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:01:27.881057 30217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:01:27.882378 30217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:01:27.894529 30223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:27.894450 30225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:27.894534 30222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:01:27.895722 30217 server_base.cc:1034] running on GCE node
I20260924 01:01:27.896512 30217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:01:27.897882 30217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:01:27.899109 30217 hybrid_clock.cc:648] HybridClock initialized: now 1790211687899063 us; error 59 us; skew 500 ppm
I20260924 01:01:27.899482 30217 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 01:01:27 dist-test-slave-mmhs krb5kdc[30204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211687, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 01:01:27.909143 30217 init.cc:377] Logged in from keytab as kudu/127.8.64.190@KRBTEST.COM (short username kudu)
I20260924 01:01:27.910840 30217 webserver.cc:460] Webserver started at http://127.8.64.190:44443/ using document root <none> and password file <none>
I20260924 01:01:27.911548 30217 fs_manager.cc:359] Metadata directory not provided
I20260924 01:01:27.911700 30217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:01:27.912014 30217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:01:27.915077 30217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "c7248f20b6ac40dc94b31fea2593eeed"
format_stamp: "Formatted at 2026-09-24 01:01:27 on dist-test-slave-mmhs"
server_key: "fe8dbb0f1fa9a9bc61afb41c2d28ccce"
server_key_iv: "15610610fda045e0cc95143d60b2c9da"
server_key_version: "encryptionkey@0"
I20260924 01:01:27.915843 30217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "c7248f20b6ac40dc94b31fea2593eeed"
format_stamp: "Formatted at 2026-09-24 01:01:27 on dist-test-slave-mmhs"
server_key: "fe8dbb0f1fa9a9bc61afb41c2d28ccce"
server_key_iv: "15610610fda045e0cc95143d60b2c9da"
server_key_version: "encryptionkey@0"
I20260924 01:01:27.921089 30217 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260924 01:01:27.924705 30232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:01:27.926045 30217 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260924 01:01:27.926237 30217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "c7248f20b6ac40dc94b31fea2593eeed"
format_stamp: "Formatted at 2026-09-24 01:01:27 on dist-test-slave-mmhs"
server_key: "fe8dbb0f1fa9a9bc61afb41c2d28ccce"
server_key_iv: "15610610fda045e0cc95143d60b2c9da"
server_key_version: "encryptionkey@0"
I20260924 01:01:27.926505 30217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:01:27.961488 30217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:01:27.965878 30217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:01:27.966202 30217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:01:27.985435 30217 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.190:37695
I20260924 01:01:27.985467 30283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.190:37695 every 8 connection(s)
I20260924 01:01:27.986861 30217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 01:01:27.987325 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 30217
I20260924 01:01:27.987515 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 01:01:27.988144 8450 external_mini_cluster.cc:1428] Setting key d4a79125358383964b859e360702e6e4
I20260924 01:01:27.991492 30284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 24 01:01:28 dist-test-slave-mmhs krb5kdc[30204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 01:01:28.003800 30284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7248f20b6ac40dc94b31fea2593eeed: Bootstrap starting.
I20260924 01:01:28.008045 30284 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7248f20b6ac40dc94b31fea2593eeed: Neither blocks nor log segments found. Creating new log.
I20260924 01:01:28.009218 30284 log.cc:824] T 00000000000000000000000000000000 P c7248f20b6ac40dc94b31fea2593eeed: Log is configured to *not* fsync() on all Append() calls
I20260924 01:01:28.012523 30284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7248f20b6ac40dc94b31fea2593eeed: No bootstrap required, opened a new log
I20260924 01:01:28.012733 30287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:27.989974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46809 (local address 127.8.64.190:37695)
0924 01:01:27.990785 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:27.990800 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:27.990847 (+ 47us) server_negotiation.cc:407] Connection header received
0924 01:01:27.993109 (+ 2262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:27.993136 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:27.993627 (+ 491us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:27.994048 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:27.995401 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:27.997501 (+ 2100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:27.998834 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:27.999822 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:28.000004 (+ 182us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:28.004067 (+ 4063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:28.004077 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:28.004101 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:01:28.004153 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:28.008224 (+ 4071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:28.009053 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:28.009061 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:28.009069 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:28.009282 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:28.009821 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:28.009827 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:28.009830 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:28.010412 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:28.010756 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:28.011138 (+ 382us) server_negotiation.cc:299] Negotiation successful
0924 01:01:28.011472 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":375,"threads_started":1}
I20260924 01:01:28.016814 8450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 01:01:28.021023 30284 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7248f20b6ac40dc94b31fea2593eeed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7248f20b6ac40dc94b31fea2593eeed" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 37695 } }
I20260924 01:01:28.021507 30284 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7248f20b6ac40dc94b31fea2593eeed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 01:01:28.021643 30284 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7248f20b6ac40dc94b31fea2593eeed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7248f20b6ac40dc94b31fea2593eeed, State: Initialized, Role: FOLLOWER
I20260924 01:01:28.022281 30284 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7248f20b6ac40dc94b31fea2593eeed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7248f20b6ac40dc94b31fea2593eeed" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 37695 } }
I20260924 01:01:28.022603 30284 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7248f20b6ac40dc94b31fea2593eeed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 01:01:28.022748 30284 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7248f20b6ac40dc94b31fea2593eeed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 01:01:28.022931 30284 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7248f20b6ac40dc94b31fea2593eeed [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:01:28.025136 30284 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7248f20b6ac40dc94b31fea2593eeed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7248f20b6ac40dc94b31fea2593eeed" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 37695 } }
I20260924 01:01:28.025627 30284 leader_election.cc:302] T 00000000000000000000000000000000 P c7248f20b6ac40dc94b31fea2593eeed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7248f20b6ac40dc94b31fea2593eeed; no voters:
I20260924 01:01:28.026314 30284 leader_election.cc:288] T 00000000000000000000000000000000 P c7248f20b6ac40dc94b31fea2593eeed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 01:01:28.026434 30294 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7248f20b6ac40dc94b31fea2593eeed [term 1 FOLLOWER]: Leader election won for term 1
I20260924 01:01:28.027702 30294 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7248f20b6ac40dc94b31fea2593eeed [term 1 LEADER]: Becoming Leader. State: Replica: c7248f20b6ac40dc94b31fea2593eeed, State: Running, Role: LEADER
I20260924 01:01:28.028301 30294 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7248f20b6ac40dc94b31fea2593eeed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7248f20b6ac40dc94b31fea2593eeed" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 37695 } }
I20260924 01:01:28.028687 30284 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7248f20b6ac40dc94b31fea2593eeed [sys.catalog]: configured and running, proceeding with master startup.
I20260924 01:01:28.030560 30296 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7248f20b6ac40dc94b31fea2593eeed [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7248f20b6ac40dc94b31fea2593eeed. Latest consensus state: current_term: 1 leader_uuid: "c7248f20b6ac40dc94b31fea2593eeed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7248f20b6ac40dc94b31fea2593eeed" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 37695 } } }
I20260924 01:01:28.030968 30296 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7248f20b6ac40dc94b31fea2593eeed [sys.catalog]: This master's current role is: LEADER
I20260924 01:01:28.031667 30295 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7248f20b6ac40dc94b31fea2593eeed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7248f20b6ac40dc94b31fea2593eeed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7248f20b6ac40dc94b31fea2593eeed" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 37695 } } }
I20260924 01:01:28.032120 30295 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7248f20b6ac40dc94b31fea2593eeed [sys.catalog]: This master's current role is: LEADER
I20260924 01:01:28.037987 30299 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 01:01:28.038849 30287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:28.021102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49435 (local address 127.8.64.190:37695)
0924 01:01:28.021306 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:28.021315 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:28.021800 (+ 485us) server_negotiation.cc:407] Connection header received
0924 01:01:28.022002 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:28.022007 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:28.022076 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:28.022175 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:28.023854 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:28.024640 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:28.025949 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:28.026591 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:28.026766 (+ 175us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:28.029602 (+ 2836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:28.029609 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:28.029614 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:01:28.029650 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:28.034623 (+ 4973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:28.035945 (+ 1322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:28.035953 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:28.035959 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:28.036214 (+ 255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:28.036952 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:28.036960 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:28.036966 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:28.037467 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:28.037859 (+ 392us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:28.038215 (+ 356us) server_negotiation.cc:299] Negotiation successful
0924 01:01:28.038437 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260924 01:01:28.049908 30299 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 01:01:28.059262 30299 catalog_manager.cc:1347] Generated new cluster ID: d01b7c897a6c4e5190d865ca240d4376
I20260924 01:01:28.059389 30299 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 01:01:28.081720 30299 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 01:01:28.083045 30299 catalog_manager.cc:1504] Loading token signing keys...
I20260924 01:01:28.093096 30299 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7248f20b6ac40dc94b31fea2593eeed: Generated new TSK 0
I20260924 01:01:28.093874 30299 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 01:01:28.100383 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 30217
I20260924 01:01:28.101504 30279 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 01:01:28.191433 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 30217
W20260924 01:01:28.209308 30202 connection.cc:535] client connection to 127.8.64.190:37695 recv error: Network error: failed to read from TLS socket (remote: 127.8.64.190:37695): Connection reset by peer (error 104)
2026-09-24T01:01:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2075 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790211343099518-8450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 01:01:28 dist-test-slave-mmhs krb5kdc[30316](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 01:01:28 dist-test-slave-mmhs krb5kdc[30316](info): set up 2 sockets
Sep 24 01:01:28 dist-test-slave-mmhs krb5kdc[30316](info): commencing operation
krb5kdc: starting...
W20260924 01:01:29.313028 8450 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.003s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Sep 24 01:01:29 dist-test-slave-mmhs krb5kdc[30316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T01:01:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T01:01:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.64.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
I20260924 01:01:29.585546 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:41515
--webserver_interface=127.8.64.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:36481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:41515
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 01:01:29.916401 30329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:01:29.916749 30329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:01:29.916913 30329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:01:29.924397 30329 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 01:01:29.924549 30329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:01:29.924678 30329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:01:29.924783 30329 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 01:01:29.924877 30329 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 01:01:29.934985 30329 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:36481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:41515
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:41515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:01:29.936312 30329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:01:29.937647 30329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:01:29.949528 30334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:29.949491 30337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:29.950028 30335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:01:29.950456 30329 server_base.cc:1034] running on GCE node
I20260924 01:01:29.951193 30329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:01:29.952541 30329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:01:29.953768 30329 hybrid_clock.cc:648] HybridClock initialized: now 1790211689953720 us; error 59 us; skew 500 ppm
I20260924 01:01:29.954108 30329 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 01:01:29 dist-test-slave-mmhs krb5kdc[30316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211689, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 01:01:29.957201 30329 init.cc:377] Logged in from keytab as kudu/127.8.64.190@KRBTEST.COM (short username kudu)
I20260924 01:01:29.958727 30329 webserver.cc:460] Webserver started at http://127.8.64.190:36839/ using document root <none> and password file <none>
I20260924 01:01:29.959403 30329 fs_manager.cc:359] Metadata directory not provided
I20260924 01:01:29.959555 30329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:01:29.959860 30329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:01:29.962873 30329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "00c014575f4a42de928de336980b3a37"
format_stamp: "Formatted at 2026-09-24 01:01:29 on dist-test-slave-mmhs"
server_key: "d45cfb214d4a3570a66d89b035d73287"
server_key_iv: "ddff6257ada0f976a70eff421125262e"
server_key_version: "encryptionkey@0"
I20260924 01:01:29.963636 30329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "00c014575f4a42de928de336980b3a37"
format_stamp: "Formatted at 2026-09-24 01:01:29 on dist-test-slave-mmhs"
server_key: "d45cfb214d4a3570a66d89b035d73287"
server_key_iv: "ddff6257ada0f976a70eff421125262e"
server_key_version: "encryptionkey@0"
I20260924 01:01:29.968899 30329 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260924 01:01:29.972409 30344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:01:29.973719 30329 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260924 01:01:29.973902 30329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "00c014575f4a42de928de336980b3a37"
format_stamp: "Formatted at 2026-09-24 01:01:29 on dist-test-slave-mmhs"
server_key: "d45cfb214d4a3570a66d89b035d73287"
server_key_iv: "ddff6257ada0f976a70eff421125262e"
server_key_version: "encryptionkey@0"
I20260924 01:01:29.974100 30329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:01:29.991577 30329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:01:29.995747 30329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:01:29.996057 30329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:01:30.015885 30329 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.190:41515
I20260924 01:01:30.015911 30395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.190:41515 every 8 connection(s)
I20260924 01:01:30.017307 30329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 01:01:30.021234 30396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:01:30.021449 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 30329
I20260924 01:01:30.021623 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 01:01:30.022163 8450 external_mini_cluster.cc:1428] Setting key fe76d10b67601f5a8c47a39a1ffd18ad
I20260924 01:01:30.035367 30396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00c014575f4a42de928de336980b3a37: Bootstrap starting.
I20260924 01:01:30.035648 30399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:30.023964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54425 (local address 127.8.64.190:41515)
0924 01:01:30.024943 (+ 979us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:30.024955 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:30.024995 (+ 40us) server_negotiation.cc:407] Connection header received
0924 01:01:30.026630 (+ 1635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:30.026703 (+ 73us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:30.027140 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:30.027461 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:30.028105 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:30.029394 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:30.031248 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:30.031899 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:30.032052 (+ 153us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:30.033315 (+ 1263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:30.033323 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:30.033335 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:01:30.033866 (+ 531us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:30.033937 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:01:30.033961 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:30.034085 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:30.034460 (+ 375us) server_negotiation.cc:299] Negotiation successful
0924 01:01:30.034643 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":752,"thread_start_us":526,"threads_started":1}
I20260924 01:01:30.038589 30396 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00c014575f4a42de928de336980b3a37: Neither blocks nor log segments found. Creating new log.
I20260924 01:01:30.039403 8450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 01:01:30.039748 30396 log.cc:824] T 00000000000000000000000000000000 P 00c014575f4a42de928de336980b3a37: Log is configured to *not* fsync() on all Append() calls
I20260924 01:01:30.042807 30396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00c014575f4a42de928de336980b3a37: No bootstrap required, opened a new log
I20260924 01:01:30.051061 30399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:30.044287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58977 (local address 127.8.64.190:41515)
0924 01:01:30.044466 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:30.044471 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:30.044967 (+ 496us) server_negotiation.cc:407] Connection header received
0924 01:01:30.045111 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:30.045117 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:30.045197 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:30.045300 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:30.045784 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:30.046983 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:30.048289 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:30.049010 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:30.049141 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:30.050212 (+ 1071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:30.050219 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:30.050223 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:01:30.050245 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:30.050317 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:01:30.050340 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:30.050464 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:30.050630 (+ 166us) server_negotiation.cc:299] Negotiation successful
0924 01:01:30.050681 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260924 01:01:30.052306 30396 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00c014575f4a42de928de336980b3a37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00c014575f4a42de928de336980b3a37" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 41515 } }
I20260924 01:01:30.052671 30396 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00c014575f4a42de928de336980b3a37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 01:01:30.052785 30396 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00c014575f4a42de928de336980b3a37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00c014575f4a42de928de336980b3a37, State: Initialized, Role: FOLLOWER
I20260924 01:01:30.053390 30396 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00c014575f4a42de928de336980b3a37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00c014575f4a42de928de336980b3a37" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 41515 } }
I20260924 01:01:30.053622 30396 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00c014575f4a42de928de336980b3a37 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 01:01:30.053771 30396 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00c014575f4a42de928de336980b3a37 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 01:01:30.053967 30396 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00c014575f4a42de928de336980b3a37 [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:01:30.056270 30396 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00c014575f4a42de928de336980b3a37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00c014575f4a42de928de336980b3a37" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 41515 } }
I20260924 01:01:30.056768 30396 leader_election.cc:302] T 00000000000000000000000000000000 P 00c014575f4a42de928de336980b3a37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00c014575f4a42de928de336980b3a37; no voters:
I20260924 01:01:30.057317 30396 leader_election.cc:288] T 00000000000000000000000000000000 P 00c014575f4a42de928de336980b3a37 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 01:01:30.057459 30406 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00c014575f4a42de928de336980b3a37 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 01:01:30.058616 30406 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00c014575f4a42de928de336980b3a37 [term 1 LEADER]: Becoming Leader. State: Replica: 00c014575f4a42de928de336980b3a37, State: Running, Role: LEADER
I20260924 01:01:30.059224 30406 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00c014575f4a42de928de336980b3a37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00c014575f4a42de928de336980b3a37" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 41515 } }
I20260924 01:01:30.059410 30396 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00c014575f4a42de928de336980b3a37 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 01:01:30.062109 30408 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00c014575f4a42de928de336980b3a37 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00c014575f4a42de928de336980b3a37. Latest consensus state: current_term: 1 leader_uuid: "00c014575f4a42de928de336980b3a37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00c014575f4a42de928de336980b3a37" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 41515 } } }
I20260924 01:01:30.062605 30408 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00c014575f4a42de928de336980b3a37 [sys.catalog]: This master's current role is: LEADER
I20260924 01:01:30.062515 30407 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00c014575f4a42de928de336980b3a37 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00c014575f4a42de928de336980b3a37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00c014575f4a42de928de336980b3a37" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 41515 } } }
I20260924 01:01:30.062825 30407 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00c014575f4a42de928de336980b3a37 [sys.catalog]: This master's current role is: LEADER
I20260924 01:01:30.063472 30414 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 01:01:30.069248 30414 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 01:01:30.078397 30414 catalog_manager.cc:1347] Generated new cluster ID: 9c5e08d5cfe240fb9b1fd9685a698064
I20260924 01:01:30.078517 30414 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 01:01:30.110986 30414 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 01:01:30.112387 30414 catalog_manager.cc:1504] Loading token signing keys...
I20260924 01:01:30.119992 30414 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00c014575f4a42de928de336980b3a37: Generated new TSK 0
I20260924 01:01:30.120770 30414 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 01:01:30.128495 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 30329
W20260924 01:01:30.130129 30391 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:54425
W20260924 01:01:30.130873 8450 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 30329: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260924 01:01:30.131001 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 30329
W20260924 01:01:30.145601 30314 connection.cc:535] client connection to 127.8.64.190:41515 recv error: Network error: failed to read from TLS socket (remote: 127.8.64.190:41515): Connection reset by peer (error 104)
2026-09-24T01:01:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1935 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790211343099518-8450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 01:01:30 dist-test-slave-mmhs krb5kdc[30427](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 01:01:30 dist-test-slave-mmhs krb5kdc[30427](info): set up 2 sockets
Sep 24 01:01:30 dist-test-slave-mmhs krb5kdc[30427](info): commencing operation
krb5kdc: starting...
W20260924 01:01:31.244624 8450 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.001s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Sep 24 01:01:31 dist-test-slave-mmhs krb5kdc[30427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T01:01:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T01:01:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.64.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
I20260924 01:01:31.522711 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:37131
--webserver_interface=127.8.64.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:36647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:37131
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 01:01:31.857465 30440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:01:31.857758 30440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:01:31.857939 30440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:01:31.865224 30440 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 01:01:31.865331 30440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:01:31.865443 30440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:01:31.865553 30440 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 01:01:31.865633 30440 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 01:01:31.875418 30440 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:36647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:37131
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:37131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:01:31.876793 30440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:01:31.878106 30440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:01:31.890022 30446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:31.890098 30448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:31.890003 30445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:01:31.890825 30440 server_base.cc:1034] running on GCE node
I20260924 01:01:31.891641 30440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:01:31.893133 30440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:01:31.894397 30440 hybrid_clock.cc:648] HybridClock initialized: now 1790211691894340 us; error 65 us; skew 500 ppm
I20260924 01:01:31.894778 30440 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 01:01:31 dist-test-slave-mmhs krb5kdc[30427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211691, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 01:01:31.898038 30440 init.cc:377] Logged in from keytab as kudu/127.8.64.190@KRBTEST.COM (short username kudu)
I20260924 01:01:31.899645 30440 webserver.cc:460] Webserver started at http://127.8.64.190:35877/ using document root <none> and password file <none>
I20260924 01:01:31.900362 30440 fs_manager.cc:359] Metadata directory not provided
I20260924 01:01:31.900514 30440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:01:31.900828 30440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:01:31.903923 30440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "0b215801e69d44b6b8ab90ca52e4d117"
format_stamp: "Formatted at 2026-09-24 01:01:31 on dist-test-slave-mmhs"
server_key: "114254af945aabfc028b7701db02580e"
server_key_iv: "719d14066862a551ac57de1efc910694"
server_key_version: "encryptionkey@0"
I20260924 01:01:31.904706 30440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "0b215801e69d44b6b8ab90ca52e4d117"
format_stamp: "Formatted at 2026-09-24 01:01:31 on dist-test-slave-mmhs"
server_key: "114254af945aabfc028b7701db02580e"
server_key_iv: "719d14066862a551ac57de1efc910694"
server_key_version: "encryptionkey@0"
I20260924 01:01:31.910228 30440 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260924 01:01:31.914006 30455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:01:31.915469 30440 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 01:01:31.915673 30440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "0b215801e69d44b6b8ab90ca52e4d117"
format_stamp: "Formatted at 2026-09-24 01:01:31 on dist-test-slave-mmhs"
server_key: "114254af945aabfc028b7701db02580e"
server_key_iv: "719d14066862a551ac57de1efc910694"
server_key_version: "encryptionkey@0"
I20260924 01:01:31.915885 30440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:01:31.921865 30440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:01:31.926168 30440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:01:31.926541 30440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:01:31.945545 30440 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.190:37131
I20260924 01:01:31.945604 30506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.190:37131 every 8 connection(s)
I20260924 01:01:31.947105 30440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 01:01:31.949479 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 30440
I20260924 01:01:31.949666 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 01:01:31.950193 8450 external_mini_cluster.cc:1428] Setting key 3b687e85be7081d628a15d2bf1287224
I20260924 01:01:31.951403 30507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:01:31.957854 30510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:31.952080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42017 (local address 127.8.64.190:37131)
0924 01:01:31.952860 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:31.952872 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:31.952911 (+ 39us) server_negotiation.cc:407] Connection header received
0924 01:01:31.954530 (+ 1619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:31.954550 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:31.954882 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:31.955269 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:31.956292 (+ 1023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:31.956298 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:31.956310 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:01:31.956325 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:31.956392 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:01:31.956410 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:31.956660 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:31.956793 (+ 133us) server_negotiation.cc:299] Negotiation successful
0924 01:01:31.956982 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":353,"threads_started":1}
I20260924 01:01:31.961489 8450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 01:01:31.963929 30507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b215801e69d44b6b8ab90ca52e4d117: Bootstrap starting.
I20260924 01:01:31.967556 30507 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b215801e69d44b6b8ab90ca52e4d117: Neither blocks nor log segments found. Creating new log.
I20260924 01:01:31.968257 30510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:31.965617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46005 (local address 127.8.64.190:37131)
0924 01:01:31.965803 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:31.965809 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:31.966352 (+ 543us) server_negotiation.cc:407] Connection header received
0924 01:01:31.966517 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:31.966522 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:31.966590 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:31.966695 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:31.967601 (+ 906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:31.967606 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:31.967612 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:01:31.967633 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:31.967679 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:01:31.967700 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:31.967825 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:31.967887 (+ 62us) server_negotiation.cc:299] Negotiation successful
0924 01:01:31.967953 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260924 01:01:31.968852 30507 log.cc:824] T 00000000000000000000000000000000 P 0b215801e69d44b6b8ab90ca52e4d117: Log is configured to *not* fsync() on all Append() calls
I20260924 01:01:31.971668 30507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b215801e69d44b6b8ab90ca52e4d117: No bootstrap required, opened a new log
I20260924 01:01:31.979560 30507 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b215801e69d44b6b8ab90ca52e4d117 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b215801e69d44b6b8ab90ca52e4d117" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 37131 } }
I20260924 01:01:31.979984 30507 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b215801e69d44b6b8ab90ca52e4d117 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 01:01:31.980098 30507 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b215801e69d44b6b8ab90ca52e4d117 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b215801e69d44b6b8ab90ca52e4d117, State: Initialized, Role: FOLLOWER
I20260924 01:01:31.980736 30507 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b215801e69d44b6b8ab90ca52e4d117 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b215801e69d44b6b8ab90ca52e4d117" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 37131 } }
I20260924 01:01:31.980988 30507 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b215801e69d44b6b8ab90ca52e4d117 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 01:01:31.981134 30507 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b215801e69d44b6b8ab90ca52e4d117 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 01:01:31.981319 30507 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b215801e69d44b6b8ab90ca52e4d117 [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:01:31.983945 30507 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b215801e69d44b6b8ab90ca52e4d117 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b215801e69d44b6b8ab90ca52e4d117" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 37131 } }
I20260924 01:01:31.984483 30507 leader_election.cc:302] T 00000000000000000000000000000000 P 0b215801e69d44b6b8ab90ca52e4d117 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b215801e69d44b6b8ab90ca52e4d117; no voters:
I20260924 01:01:31.985396 30507 leader_election.cc:288] T 00000000000000000000000000000000 P 0b215801e69d44b6b8ab90ca52e4d117 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 01:01:31.985675 30517 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b215801e69d44b6b8ab90ca52e4d117 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 01:01:31.986418 30517 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b215801e69d44b6b8ab90ca52e4d117 [term 1 LEADER]: Becoming Leader. State: Replica: 0b215801e69d44b6b8ab90ca52e4d117, State: Running, Role: LEADER
I20260924 01:01:31.987066 30517 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b215801e69d44b6b8ab90ca52e4d117 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b215801e69d44b6b8ab90ca52e4d117" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 37131 } }
I20260924 01:01:31.987726 30507 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b215801e69d44b6b8ab90ca52e4d117 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 01:01:31.990391 30518 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b215801e69d44b6b8ab90ca52e4d117 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b215801e69d44b6b8ab90ca52e4d117" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b215801e69d44b6b8ab90ca52e4d117" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 37131 } } }
I20260924 01:01:31.990408 30519 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b215801e69d44b6b8ab90ca52e4d117 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b215801e69d44b6b8ab90ca52e4d117. Latest consensus state: current_term: 1 leader_uuid: "0b215801e69d44b6b8ab90ca52e4d117" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b215801e69d44b6b8ab90ca52e4d117" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 37131 } } }
I20260924 01:01:31.990911 30518 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b215801e69d44b6b8ab90ca52e4d117 [sys.catalog]: This master's current role is: LEADER
I20260924 01:01:31.990965 30519 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b215801e69d44b6b8ab90ca52e4d117 [sys.catalog]: This master's current role is: LEADER
I20260924 01:01:31.991976 30525 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 01:01:31.998358 30525 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 01:01:32.008615 30525 catalog_manager.cc:1347] Generated new cluster ID: b4eade4c86a14c6ca6a2cc50313e75f1
I20260924 01:01:32.008769 30525 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 01:01:32.024219 30525 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 01:01:32.024637 30525 catalog_manager.cc:1504] Loading token signing keys...
I20260924 01:01:32.043478 30525 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b215801e69d44b6b8ab90ca52e4d117: Generated new TSK 0
I20260924 01:01:32.044421 30525 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 01:01:32.055548 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 30440
W20260924 01:01:32.056864 30502 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:42017
W20260924 01:01:32.057632 8450 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 30440: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260924 01:01:32.057790 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 30440
2026-09-24T01:01:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1927 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790211343099518-8450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 01:01:32 dist-test-slave-mmhs krb5kdc[30538](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 01:01:32 dist-test-slave-mmhs krb5kdc[30538](info): set up 2 sockets
Sep 24 01:01:32 dist-test-slave-mmhs krb5kdc[30538](info): commencing operation
krb5kdc: starting...
W20260924 01:01:33.180390 8450 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.004s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Sep 24 01:01:33 dist-test-slave-mmhs krb5kdc[30538](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T01:01:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T01:01:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.64.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
I20260924 01:01:33.469806 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:44969
--webserver_interface=127.8.64.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:35639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:44969
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 01:01:33.803582 30551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:01:33.803869 30551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:01:33.804040 30551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:01:33.811322 30551 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 01:01:33.811432 30551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:01:33.811515 30551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:01:33.811630 30551 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 01:01:33.811697 30551 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 01:01:33.821630 30551 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:35639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:44969
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:44969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:01:33.823016 30551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:01:33.824376 30551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:01:33.835934 30559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:33.836095 30556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:33.836061 30557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:01:33.837278 30551 server_base.cc:1034] running on GCE node
I20260924 01:01:33.838124 30551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:01:33.839555 30551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:01:33.840759 30551 hybrid_clock.cc:648] HybridClock initialized: now 1790211693840714 us; error 44 us; skew 500 ppm
I20260924 01:01:33.841164 30551 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 01:01:33 dist-test-slave-mmhs krb5kdc[30538](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211693, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 01:01:33.851154 30551 init.cc:377] Logged in from keytab as kudu/127.8.64.190@KRBTEST.COM (short username kudu)
I20260924 01:01:33.852876 30551 webserver.cc:460] Webserver started at http://127.8.64.190:38013/ using document root <none> and password file <none>
I20260924 01:01:33.853605 30551 fs_manager.cc:359] Metadata directory not provided
I20260924 01:01:33.853740 30551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:01:33.854074 30551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:01:33.857221 30551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "486be483eaba49e2a27126378b1548db"
format_stamp: "Formatted at 2026-09-24 01:01:33 on dist-test-slave-mmhs"
server_key: "15f4b1ab410ed34c0062b22f83e26f4e"
server_key_iv: "6a34ed5415b704d95a2ab4b452bcc912"
server_key_version: "encryptionkey@0"
I20260924 01:01:33.858003 30551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "486be483eaba49e2a27126378b1548db"
format_stamp: "Formatted at 2026-09-24 01:01:33 on dist-test-slave-mmhs"
server_key: "15f4b1ab410ed34c0062b22f83e26f4e"
server_key_iv: "6a34ed5415b704d95a2ab4b452bcc912"
server_key_version: "encryptionkey@0"
I20260924 01:01:33.863571 30551 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260924 01:01:33.867126 30566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:01:33.868495 30551 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260924 01:01:33.868680 30551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "486be483eaba49e2a27126378b1548db"
format_stamp: "Formatted at 2026-09-24 01:01:33 on dist-test-slave-mmhs"
server_key: "15f4b1ab410ed34c0062b22f83e26f4e"
server_key_iv: "6a34ed5415b704d95a2ab4b452bcc912"
server_key_version: "encryptionkey@0"
I20260924 01:01:33.868867 30551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:01:33.889752 30551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:01:33.894201 30551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:01:33.894522 30551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:01:33.913480 30551 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.190:44969
I20260924 01:01:33.913517 30617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.190:44969 every 8 connection(s)
I20260924 01:01:33.914980 30551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 01:01:33.917193 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 30551
I20260924 01:01:33.917392 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 01:01:33.918035 8450 external_mini_cluster.cc:1428] Setting key 3fde9b816b24f9662a489805a9c84564
I20260924 01:01:33.919785 30618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 24 01:01:33 dist-test-slave-mmhs krb5kdc[30538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 01:01:33.933658 30618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 486be483eaba49e2a27126378b1548db: Bootstrap starting.
I20260924 01:01:33.936887 30618 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 486be483eaba49e2a27126378b1548db: Neither blocks nor log segments found. Creating new log.
I20260924 01:01:33.937909 30618 log.cc:824] T 00000000000000000000000000000000 P 486be483eaba49e2a27126378b1548db: Log is configured to *not* fsync() on all Append() calls
I20260924 01:01:33.940796 30618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 486be483eaba49e2a27126378b1548db: No bootstrap required, opened a new log
I20260924 01:01:33.941011 30621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:33.919877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34537 (local address 127.8.64.190:44969)
0924 01:01:33.920690 (+ 813us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:33.920701 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:33.920741 (+ 40us) server_negotiation.cc:407] Connection header received
0924 01:01:33.922438 (+ 1697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:33.922459 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:33.922884 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:33.923213 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:33.925044 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:33.926482 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:33.928255 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:33.928924 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:33.929048 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:33.933748 (+ 4700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:33.933755 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:33.933770 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:01:33.933842 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:33.936977 (+ 3135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:33.937675 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:33.937685 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:33.937698 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:33.937878 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:33.938280 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:33.938285 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:33.938290 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:33.938879 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:33.939221 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:33.939587 (+ 366us) server_negotiation.cc:299] Negotiation successful
0924 01:01:33.940029 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":367,"threads_started":1}
I20260924 01:01:33.944638 8450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 01:01:33.948675 30618 raft_consensus.cc:348] T 00000000000000000000000000000000 P 486be483eaba49e2a27126378b1548db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "486be483eaba49e2a27126378b1548db" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44969 } }
I20260924 01:01:33.949133 30618 raft_consensus.cc:374] T 00000000000000000000000000000000 P 486be483eaba49e2a27126378b1548db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 01:01:33.949254 30618 raft_consensus.cc:729] T 00000000000000000000000000000000 P 486be483eaba49e2a27126378b1548db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 486be483eaba49e2a27126378b1548db, State: Initialized, Role: FOLLOWER
I20260924 01:01:33.949916 30618 consensus_queue.cc:260] T 00000000000000000000000000000000 P 486be483eaba49e2a27126378b1548db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "486be483eaba49e2a27126378b1548db" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44969 } }
I20260924 01:01:33.950222 30618 raft_consensus.cc:388] T 00000000000000000000000000000000 P 486be483eaba49e2a27126378b1548db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 01:01:33.950415 30618 raft_consensus.cc:482] T 00000000000000000000000000000000 P 486be483eaba49e2a27126378b1548db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 01:01:33.950623 30618 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 486be483eaba49e2a27126378b1548db [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:01:33.953595 30618 raft_consensus.cc:504] T 00000000000000000000000000000000 P 486be483eaba49e2a27126378b1548db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "486be483eaba49e2a27126378b1548db" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44969 } }
I20260924 01:01:33.954142 30618 leader_election.cc:302] T 00000000000000000000000000000000 P 486be483eaba49e2a27126378b1548db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 486be483eaba49e2a27126378b1548db; no voters:
I20260924 01:01:33.954803 30618 leader_election.cc:288] T 00000000000000000000000000000000 P 486be483eaba49e2a27126378b1548db [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 01:01:33.954942 30628 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 486be483eaba49e2a27126378b1548db [term 1 FOLLOWER]: Leader election won for term 1
I20260924 01:01:33.956161 30628 raft_consensus.cc:686] T 00000000000000000000000000000000 P 486be483eaba49e2a27126378b1548db [term 1 LEADER]: Becoming Leader. State: Replica: 486be483eaba49e2a27126378b1548db, State: Running, Role: LEADER
I20260924 01:01:33.956919 30618 sys_catalog.cc:564] T 00000000000000000000000000000000 P 486be483eaba49e2a27126378b1548db [sys.catalog]: configured and running, proceeding with master startup.
I20260924 01:01:33.957135 30628 consensus_queue.cc:237] T 00000000000000000000000000000000 P 486be483eaba49e2a27126378b1548db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "486be483eaba49e2a27126378b1548db" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44969 } }
I20260924 01:01:33.966992 30629 sys_catalog.cc:455] T 00000000000000000000000000000000 P 486be483eaba49e2a27126378b1548db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "486be483eaba49e2a27126378b1548db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "486be483eaba49e2a27126378b1548db" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44969 } } }
I20260924 01:01:33.966966 30630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 486be483eaba49e2a27126378b1548db [sys.catalog]: SysCatalogTable state changed. Reason: New leader 486be483eaba49e2a27126378b1548db. Latest consensus state: current_term: 1 leader_uuid: "486be483eaba49e2a27126378b1548db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "486be483eaba49e2a27126378b1548db" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44969 } } }
I20260924 01:01:33.967417 30629 sys_catalog.cc:458] T 00000000000000000000000000000000 P 486be483eaba49e2a27126378b1548db [sys.catalog]: This master's current role is: LEADER
I20260924 01:01:33.967515 30630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 486be483eaba49e2a27126378b1548db [sys.catalog]: This master's current role is: LEADER
I20260924 01:01:33.968528 30621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:33.949231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53121 (local address 127.8.64.190:44969)
0924 01:01:33.949400 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:33.949406 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:33.949926 (+ 520us) server_negotiation.cc:407] Connection header received
0924 01:01:33.950090 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:33.950093 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:33.950187 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:33.950275 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:33.952100 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:33.952827 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:33.954546 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:33.955424 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:33.955594 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:33.958994 (+ 3400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:33.959003 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:33.959011 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:01:33.959076 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:33.963877 (+ 4801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:33.964930 (+ 1053us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:33.964937 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:33.964941 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:33.965291 (+ 350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:33.965833 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:33.965838 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:33.965841 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:33.966260 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:33.967506 (+ 1246us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:33.967698 (+ 192us) server_negotiation.cc:299] Negotiation successful
0924 01:01:33.967931 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260924 01:01:33.970170 30642 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 01:01:33.974509 30642 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 01:01:33.982906 30642 catalog_manager.cc:1347] Generated new cluster ID: fb5a6ce449e643f6bec8f1cbf3a630db
I20260924 01:01:33.983017 30642 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 01:01:33.995934 30642 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 01:01:33.997295 30642 catalog_manager.cc:1504] Loading token signing keys...
I20260924 01:01:34.008982 30642 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 486be483eaba49e2a27126378b1548db: Generated new TSK 0
I20260924 01:01:34.009812 30642 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 01:01:34.016294 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 30551
I20260924 01:01:34.017441 30613 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 01:01:34.106530 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 30551
2026-09-24T01:01:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2051 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790211343099518-8450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 01:01:34 dist-test-slave-mmhs krb5kdc[30650](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 01:01:34 dist-test-slave-mmhs krb5kdc[30650](info): set up 2 sockets
Sep 24 01:01:34 dist-test-slave-mmhs krb5kdc[30650](info): commencing operation
krb5kdc: starting...
W20260924 01:01:35.216491 8450 mini_kdc.cc:121] Time spent starting KDC: real 1.073s user 0.001s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Sep 24 01:01:35 dist-test-slave-mmhs krb5kdc[30650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211695, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T01:01:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T01:01:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.64.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
I20260924 01:01:35.486395 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:44241
--webserver_interface=127.8.64.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:36393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:44241
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 01:01:35.816139 30663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:01:35.816475 30663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:01:35.816655 30663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:01:35.823920 30663 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 01:01:35.824080 30663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:01:35.824187 30663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:01:35.824294 30663 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 01:01:35.824369 30663 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 01:01:35.834178 30663 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:36393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:44241
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:44241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:01:35.835529 30663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:01:35.836846 30663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:01:35.848774 30668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:35.848753 30669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:35.849129 30671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:01:35.849594 30663 server_base.cc:1034] running on GCE node
I20260924 01:01:35.850430 30663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:01:35.851776 30663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:01:35.852973 30663 hybrid_clock.cc:648] HybridClock initialized: now 1790211695852924 us; error 44 us; skew 500 ppm
I20260924 01:01:35.853313 30663 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 01:01:35 dist-test-slave-mmhs krb5kdc[30650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211695, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 01:01:35.856436 30663 init.cc:377] Logged in from keytab as kudu/127.8.64.190@KRBTEST.COM (short username kudu)
I20260924 01:01:35.857899 30663 webserver.cc:460] Webserver started at http://127.8.64.190:42411/ using document root <none> and password file <none>
I20260924 01:01:35.858632 30663 fs_manager.cc:359] Metadata directory not provided
I20260924 01:01:35.858770 30663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:01:35.859076 30663 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:01:35.861989 30663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "e8a8f1b6b65747fd9f8995a53a57d69f"
format_stamp: "Formatted at 2026-09-24 01:01:35 on dist-test-slave-mmhs"
server_key: "a744f567de807d36e9ae43ee83d4d825"
server_key_iv: "c559ff846f78acf90994bddc6491dc8e"
server_key_version: "encryptionkey@0"
I20260924 01:01:35.862795 30663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "e8a8f1b6b65747fd9f8995a53a57d69f"
format_stamp: "Formatted at 2026-09-24 01:01:35 on dist-test-slave-mmhs"
server_key: "a744f567de807d36e9ae43ee83d4d825"
server_key_iv: "c559ff846f78acf90994bddc6491dc8e"
server_key_version: "encryptionkey@0"
I20260924 01:01:35.867724 30663 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260924 01:01:35.871115 30678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:01:35.872349 30663 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260924 01:01:35.872534 30663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "e8a8f1b6b65747fd9f8995a53a57d69f"
format_stamp: "Formatted at 2026-09-24 01:01:35 on dist-test-slave-mmhs"
server_key: "a744f567de807d36e9ae43ee83d4d825"
server_key_iv: "c559ff846f78acf90994bddc6491dc8e"
server_key_version: "encryptionkey@0"
I20260924 01:01:35.872720 30663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:01:35.910619 30663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:01:35.915179 30663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:01:35.915493 30663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:01:35.934912 30663 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.190:44241
I20260924 01:01:35.934953 30729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.190:44241 every 8 connection(s)
I20260924 01:01:35.936332 30663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 01:01:35.940025 30730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:01:35.942657 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 30663
I20260924 01:01:35.942853 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 01:01:35.943341 8450 external_mini_cluster.cc:1428] Setting key 8d6edf4df4aa571cc38469c4a9fef20f
I20260924 01:01:35.951709 30730 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8a8f1b6b65747fd9f8995a53a57d69f: Bootstrap starting.
I20260924 01:01:35.955425 30730 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8a8f1b6b65747fd9f8995a53a57d69f: Neither blocks nor log segments found. Creating new log.
I20260924 01:01:35.956574 30730 log.cc:824] T 00000000000000000000000000000000 P e8a8f1b6b65747fd9f8995a53a57d69f: Log is configured to *not* fsync() on all Append() calls
I20260924 01:01:35.956624 30733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:35.945077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54329 (local address 127.8.64.190:44241)
0924 01:01:35.945853 (+ 776us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:35.945864 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:35.945905 (+ 41us) server_negotiation.cc:407] Connection header received
0924 01:01:35.947619 (+ 1714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:35.947641 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:35.948037 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:35.948387 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:35.948988 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:35.950378 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:35.952188 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:35.952826 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:35.952961 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:35.954333 (+ 1372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:35.954345 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:35.954366 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:01:35.954875 (+ 509us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:35.954948 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:01:35.954971 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:35.955063 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:35.955371 (+ 308us) server_negotiation.cc:299] Negotiation successful
0924 01:01:35.955672 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":380,"threads_started":1}
I20260924 01:01:35.959542 30730 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8a8f1b6b65747fd9f8995a53a57d69f: No bootstrap required, opened a new log
I20260924 01:01:35.960220 8450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 01:01:35.968174 30730 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8a8f1b6b65747fd9f8995a53a57d69f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8a8f1b6b65747fd9f8995a53a57d69f" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44241 } }
I20260924 01:01:35.968633 30730 raft_consensus.cc:374] T 00000000000000000000000000000000 P e8a8f1b6b65747fd9f8995a53a57d69f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 01:01:35.968732 30730 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8a8f1b6b65747fd9f8995a53a57d69f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8a8f1b6b65747fd9f8995a53a57d69f, State: Initialized, Role: FOLLOWER
I20260924 01:01:35.969437 30730 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8a8f1b6b65747fd9f8995a53a57d69f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8a8f1b6b65747fd9f8995a53a57d69f" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44241 } }
I20260924 01:01:35.969738 30730 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8a8f1b6b65747fd9f8995a53a57d69f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 01:01:35.969914 30730 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8a8f1b6b65747fd9f8995a53a57d69f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 01:01:35.970124 30730 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8a8f1b6b65747fd9f8995a53a57d69f [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:01:35.971880 30733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:35.964459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51029 (local address 127.8.64.190:44241)
0924 01:01:35.964648 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:35.964654 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:35.965146 (+ 492us) server_negotiation.cc:407] Connection header received
0924 01:01:35.965293 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:35.965298 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:35.965378 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:35.965479 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:35.966053 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:35.967219 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:35.969021 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:35.969716 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:35.969851 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:35.971069 (+ 1218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:35.971076 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:35.971081 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:01:35.971102 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:35.971149 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:01:35.971168 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:35.971270 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:35.971425 (+ 155us) server_negotiation.cc:299] Negotiation successful
0924 01:01:35.971549 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260924 01:01:35.972879 30730 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8a8f1b6b65747fd9f8995a53a57d69f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8a8f1b6b65747fd9f8995a53a57d69f" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44241 } }
I20260924 01:01:35.973387 30730 leader_election.cc:302] T 00000000000000000000000000000000 P e8a8f1b6b65747fd9f8995a53a57d69f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8a8f1b6b65747fd9f8995a53a57d69f; no voters:
I20260924 01:01:35.973982 30730 leader_election.cc:288] T 00000000000000000000000000000000 P e8a8f1b6b65747fd9f8995a53a57d69f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 01:01:35.974123 30740 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8a8f1b6b65747fd9f8995a53a57d69f [term 1 FOLLOWER]: Leader election won for term 1
I20260924 01:01:35.974948 30740 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8a8f1b6b65747fd9f8995a53a57d69f [term 1 LEADER]: Becoming Leader. State: Replica: e8a8f1b6b65747fd9f8995a53a57d69f, State: Running, Role: LEADER
I20260924 01:01:35.975625 30740 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8a8f1b6b65747fd9f8995a53a57d69f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8a8f1b6b65747fd9f8995a53a57d69f" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44241 } }
I20260924 01:01:35.976141 30730 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8a8f1b6b65747fd9f8995a53a57d69f [sys.catalog]: configured and running, proceeding with master startup.
I20260924 01:01:35.979025 30741 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8a8f1b6b65747fd9f8995a53a57d69f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e8a8f1b6b65747fd9f8995a53a57d69f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8a8f1b6b65747fd9f8995a53a57d69f" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44241 } } }
I20260924 01:01:35.979249 30742 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8a8f1b6b65747fd9f8995a53a57d69f [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8a8f1b6b65747fd9f8995a53a57d69f. Latest consensus state: current_term: 1 leader_uuid: "e8a8f1b6b65747fd9f8995a53a57d69f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8a8f1b6b65747fd9f8995a53a57d69f" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 44241 } } }
I20260924 01:01:35.979575 30741 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8a8f1b6b65747fd9f8995a53a57d69f [sys.catalog]: This master's current role is: LEADER
I20260924 01:01:35.979583 30742 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8a8f1b6b65747fd9f8995a53a57d69f [sys.catalog]: This master's current role is: LEADER
I20260924 01:01:35.983343 30746 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 01:01:35.989122 30746 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 01:01:35.998656 30746 catalog_manager.cc:1347] Generated new cluster ID: 06c79a92ca3b45e0972f119687a9c8fa
I20260924 01:01:35.998781 30746 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 01:01:36.015532 30746 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 01:01:36.017186 30746 catalog_manager.cc:1504] Loading token signing keys...
I20260924 01:01:36.030200 30746 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e8a8f1b6b65747fd9f8995a53a57d69f: Generated new TSK 0
I20260924 01:01:36.031030 30746 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 01:01:36.038283 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 30663
W20260924 01:01:36.039510 30725 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:54329
W20260924 01:01:36.040179 8450 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 30663: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260924 01:01:36.040364 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 30663
2026-09-24T01:01:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1933 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790211343099518-8450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 01:01:36 dist-test-slave-mmhs krb5kdc[30761](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 01:01:36 dist-test-slave-mmhs krb5kdc[30761](info): set up 2 sockets
Sep 24 01:01:36 dist-test-slave-mmhs krb5kdc[30761](info): commencing operation
krb5kdc: starting...
W20260924 01:01:37.164543 8450 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.001s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Sep 24 01:01:37 dist-test-slave-mmhs krb5kdc[30761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T01:01:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T01:01:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.64.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
I20260924 01:01:37.450220 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:45073
--webserver_interface=127.8.64.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:36833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:45073
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 01:01:37.783125 30774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:01:37.783420 30774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:01:37.783512 30774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:01:37.790688 30774 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 01:01:37.790803 30774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:01:37.790858 30774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:01:37.790920 30774 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 01:01:37.790963 30774 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 01:01:37.800786 30774 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:36833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:45073
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:45073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:01:37.802037 30774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:01:37.803300 30774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:01:37.815231 30782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:37.815202 30779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:37.815270 30780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:01:37.816078 30774 server_base.cc:1034] running on GCE node
I20260924 01:01:37.816846 30774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:01:37.818259 30774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:01:37.819535 30774 hybrid_clock.cc:648] HybridClock initialized: now 1790211697819490 us; error 29 us; skew 500 ppm
I20260924 01:01:37.819902 30774 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 01:01:37 dist-test-slave-mmhs krb5kdc[30761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211697, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 01:01:37.823172 30774 init.cc:377] Logged in from keytab as kudu/127.8.64.190@KRBTEST.COM (short username kudu)
I20260924 01:01:37.824779 30774 webserver.cc:460] Webserver started at http://127.8.64.190:34785/ using document root <none> and password file <none>
I20260924 01:01:37.825446 30774 fs_manager.cc:359] Metadata directory not provided
I20260924 01:01:37.825654 30774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:01:37.825950 30774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:01:37.829037 30774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "e504d2155baf4acaae6e68d82b53fddb"
format_stamp: "Formatted at 2026-09-24 01:01:37 on dist-test-slave-mmhs"
server_key: "34c89a16e9fb1dbf504725ab0b12ca4e"
server_key_iv: "03681ca7cd9ff4a7ee62894fea6707db"
server_key_version: "encryptionkey@0"
I20260924 01:01:37.829768 30774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "e504d2155baf4acaae6e68d82b53fddb"
format_stamp: "Formatted at 2026-09-24 01:01:37 on dist-test-slave-mmhs"
server_key: "34c89a16e9fb1dbf504725ab0b12ca4e"
server_key_iv: "03681ca7cd9ff4a7ee62894fea6707db"
server_key_version: "encryptionkey@0"
I20260924 01:01:37.835098 30774 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260924 01:01:37.838688 30789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:01:37.840022 30774 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260924 01:01:37.840164 30774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "e504d2155baf4acaae6e68d82b53fddb"
format_stamp: "Formatted at 2026-09-24 01:01:37 on dist-test-slave-mmhs"
server_key: "34c89a16e9fb1dbf504725ab0b12ca4e"
server_key_iv: "03681ca7cd9ff4a7ee62894fea6707db"
server_key_version: "encryptionkey@0"
I20260924 01:01:37.840325 30774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:01:37.845969 30774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:01:37.849952 30774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:01:37.850205 30774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:01:37.868815 30774 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.190:45073
I20260924 01:01:37.868866 30840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.190:45073 every 8 connection(s)
I20260924 01:01:37.870247 30774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 01:01:37.874020 30841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:01:37.876865 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 30774
I20260924 01:01:37.877035 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 01:01:37.877552 8450 external_mini_cluster.cc:1428] Setting key 1ee2b03cc3d137957a6d0f812138e064
I20260924 01:01:37.885221 30844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:37.879373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40019 (local address 127.8.64.190:45073)
0924 01:01:37.880214 (+ 841us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:37.880226 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:37.880267 (+ 41us) server_negotiation.cc:407] Connection header received
0924 01:01:37.881904 (+ 1637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:37.881926 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:37.882259 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:37.882671 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:37.883709 (+ 1038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:37.883715 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:37.883727 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:01:37.883744 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:37.883807 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:01:37.883830 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:37.884004 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:37.884113 (+ 109us) server_negotiation.cc:299] Negotiation successful
0924 01:01:37.884318 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":405,"threads_started":1}
I20260924 01:01:37.886451 30841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e504d2155baf4acaae6e68d82b53fddb: Bootstrap starting.
I20260924 01:01:37.889153 8450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 01:01:37.889887 30841 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e504d2155baf4acaae6e68d82b53fddb: Neither blocks nor log segments found. Creating new log.
I20260924 01:01:37.890985 30841 log.cc:824] T 00000000000000000000000000000000 P e504d2155baf4acaae6e68d82b53fddb: Log is configured to *not* fsync() on all Append() calls
I20260924 01:01:37.894635 30841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e504d2155baf4acaae6e68d82b53fddb: No bootstrap required, opened a new log
I20260924 01:01:37.896282 30844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:37.893603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44735 (local address 127.8.64.190:45073)
0924 01:01:37.893822 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:37.893829 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:37.894222 (+ 393us) server_negotiation.cc:407] Connection header received
0924 01:01:37.894406 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:37.894412 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:37.894476 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:37.894584 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:37.895540 (+ 956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:37.895547 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:37.895551 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 01:01:37.895570 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:37.895620 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 01:01:37.895640 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:37.895761 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:37.895884 (+ 123us) server_negotiation.cc:299] Negotiation successful
0924 01:01:37.895949 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260924 01:01:37.902758 30841 raft_consensus.cc:348] T 00000000000000000000000000000000 P e504d2155baf4acaae6e68d82b53fddb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e504d2155baf4acaae6e68d82b53fddb" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45073 } }
I20260924 01:01:37.903151 30841 raft_consensus.cc:374] T 00000000000000000000000000000000 P e504d2155baf4acaae6e68d82b53fddb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 01:01:37.903268 30841 raft_consensus.cc:729] T 00000000000000000000000000000000 P e504d2155baf4acaae6e68d82b53fddb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e504d2155baf4acaae6e68d82b53fddb, State: Initialized, Role: FOLLOWER
I20260924 01:01:37.903825 30841 consensus_queue.cc:260] T 00000000000000000000000000000000 P e504d2155baf4acaae6e68d82b53fddb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e504d2155baf4acaae6e68d82b53fddb" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45073 } }
I20260924 01:01:37.904071 30841 raft_consensus.cc:388] T 00000000000000000000000000000000 P e504d2155baf4acaae6e68d82b53fddb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 01:01:37.904219 30841 raft_consensus.cc:482] T 00000000000000000000000000000000 P e504d2155baf4acaae6e68d82b53fddb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 01:01:37.904398 30841 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e504d2155baf4acaae6e68d82b53fddb [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:01:37.906697 30841 raft_consensus.cc:504] T 00000000000000000000000000000000 P e504d2155baf4acaae6e68d82b53fddb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e504d2155baf4acaae6e68d82b53fddb" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45073 } }
I20260924 01:01:37.907181 30841 leader_election.cc:302] T 00000000000000000000000000000000 P e504d2155baf4acaae6e68d82b53fddb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e504d2155baf4acaae6e68d82b53fddb; no voters:
I20260924 01:01:37.907810 30841 leader_election.cc:288] T 00000000000000000000000000000000 P e504d2155baf4acaae6e68d82b53fddb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 01:01:37.908015 30851 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e504d2155baf4acaae6e68d82b53fddb [term 1 FOLLOWER]: Leader election won for term 1
I20260924 01:01:37.908850 30851 raft_consensus.cc:686] T 00000000000000000000000000000000 P e504d2155baf4acaae6e68d82b53fddb [term 1 LEADER]: Becoming Leader. State: Replica: e504d2155baf4acaae6e68d82b53fddb, State: Running, Role: LEADER
I20260924 01:01:37.909467 30851 consensus_queue.cc:237] T 00000000000000000000000000000000 P e504d2155baf4acaae6e68d82b53fddb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e504d2155baf4acaae6e68d82b53fddb" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45073 } }
I20260924 01:01:37.909951 30841 sys_catalog.cc:564] T 00000000000000000000000000000000 P e504d2155baf4acaae6e68d82b53fddb [sys.catalog]: configured and running, proceeding with master startup.
I20260924 01:01:37.913084 30852 sys_catalog.cc:455] T 00000000000000000000000000000000 P e504d2155baf4acaae6e68d82b53fddb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e504d2155baf4acaae6e68d82b53fddb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e504d2155baf4acaae6e68d82b53fddb" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45073 } } }
I20260924 01:01:37.914239 30852 sys_catalog.cc:458] T 00000000000000000000000000000000 P e504d2155baf4acaae6e68d82b53fddb [sys.catalog]: This master's current role is: LEADER
I20260924 01:01:37.913661 30853 sys_catalog.cc:455] T 00000000000000000000000000000000 P e504d2155baf4acaae6e68d82b53fddb [sys.catalog]: SysCatalogTable state changed. Reason: New leader e504d2155baf4acaae6e68d82b53fddb. Latest consensus state: current_term: 1 leader_uuid: "e504d2155baf4acaae6e68d82b53fddb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e504d2155baf4acaae6e68d82b53fddb" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45073 } } }
I20260924 01:01:37.914434 30853 sys_catalog.cc:458] T 00000000000000000000000000000000 P e504d2155baf4acaae6e68d82b53fddb [sys.catalog]: This master's current role is: LEADER
I20260924 01:01:37.916109 30859 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 01:01:37.923765 30859 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 01:01:37.932466 30859 catalog_manager.cc:1347] Generated new cluster ID: b597a078d332460fb8a34aa76f2c9870
I20260924 01:01:37.932600 30859 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 01:01:37.950340 30859 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 01:01:37.950886 30859 catalog_manager.cc:1504] Loading token signing keys...
I20260924 01:01:37.958248 30859 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e504d2155baf4acaae6e68d82b53fddb: Generated new TSK 0
I20260924 01:01:37.959242 30859 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 01:01:37.965159 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 30774
W20260924 01:01:37.966228 30836 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:40019
W20260924 01:01:37.966941 8450 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 30774: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260924 01:01:37.967087 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 30774
2026-09-24T01:01:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1927 ms)
[----------] 18 tests from AuthTokenIssuingTest (35455 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 01:01:38 dist-test-slave-mmhs krb5kdc[30872](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 01:01:38 dist-test-slave-mmhs krb5kdc[30872](info): set up 2 sockets
Sep 24 01:01:38 dist-test-slave-mmhs krb5kdc[30872](info): commencing operation
krb5kdc: starting...
W20260924 01:01:39.189705 8450 mini_kdc.cc:121] Time spent starting KDC: real 1.185s user 0.003s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Sep 24 01:01:39 dist-test-slave-mmhs krb5kdc[30872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T01:01:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T01:01:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.64.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
I20260924 01:01:39.470991 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:43183
--webserver_interface=127.8.64.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:40473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:43183,127.8.64.189:43859,127.8.64.188:36541
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 01:01:39.806058 30885 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 01:01:39.806461 30885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:01:39.806588 30885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:01:39.806728 30885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:01:39.814150 30885 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 01:01:39.814244 30885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:01:39.814353 30885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:01:39.814437 30885 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 01:01:39.814551 30885 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 01:01:39.825350 30885 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:40473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:43183,127.8.64.189:43859,127.8.64.188:36541
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:43183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:01:39.826740 30885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:01:39.828274 30885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:01:39.841096 30890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:39.841153 30891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:39.841257 30893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:01:39.841873 30885 server_base.cc:1034] running on GCE node
I20260924 01:01:39.842746 30885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:01:39.844218 30885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:01:39.845466 30885 hybrid_clock.cc:648] HybridClock initialized: now 1790211699845395 us; error 48 us; skew 500 ppm
I20260924 01:01:39.845808 30885 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 01:01:39 dist-test-slave-mmhs krb5kdc[30872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211699, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 01:01:39.852991 30885 init.cc:377] Logged in from keytab as kudu/127.8.64.190@KRBTEST.COM (short username kudu)
I20260924 01:01:39.854617 30885 webserver.cc:460] Webserver started at http://127.8.64.190:38643/ using document root <none> and password file <none>
I20260924 01:01:39.855369 30885 fs_manager.cc:359] Metadata directory not provided
I20260924 01:01:39.855501 30885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:01:39.855831 30885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:01:39.858886 30885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "b90c9c5c753d4c8ab5e279460aaa7c06"
format_stamp: "Formatted at 2026-09-24 01:01:39 on dist-test-slave-mmhs"
server_key: "0705751862714f425d757a915f273d0a"
server_key_iv: "81639910e66844fbf5fe8792cdecfdeb"
server_key_version: "encryptionkey@0"
I20260924 01:01:39.859696 30885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "b90c9c5c753d4c8ab5e279460aaa7c06"
format_stamp: "Formatted at 2026-09-24 01:01:39 on dist-test-slave-mmhs"
server_key: "0705751862714f425d757a915f273d0a"
server_key_iv: "81639910e66844fbf5fe8792cdecfdeb"
server_key_version: "encryptionkey@0"
I20260924 01:01:39.865062 30885 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260924 01:01:39.868480 30900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:01:39.869859 30885 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260924 01:01:39.870052 30885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "b90c9c5c753d4c8ab5e279460aaa7c06"
format_stamp: "Formatted at 2026-09-24 01:01:39 on dist-test-slave-mmhs"
server_key: "0705751862714f425d757a915f273d0a"
server_key_iv: "81639910e66844fbf5fe8792cdecfdeb"
server_key_version: "encryptionkey@0"
I20260924 01:01:39.870275 30885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:01:39.916990 30885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:01:39.922418 30885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:01:39.922710 30885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:01:39.942420 30885 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.190:43183
I20260924 01:01:39.942440 30951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.190:43183 every 8 connection(s)
I20260924 01:01:39.943832 30885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 01:01:39.947595 30952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:01:39.948499 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 30885
I20260924 01:01:39.948714 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 01:01:39.949379 8450 external_mini_cluster.cc:1428] Setting key 2d2f5f32485b6568775f50bb750d1720
I20260924 01:01:39.954156 30952 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.64.190" port: 43183 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 01:01:39 dist-test-slave-mmhs krb5kdc[30872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211699, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.64.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.189@KRBTEST.COM" created.
I20260924 01:01:39.979817 30954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:39.957404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53604 (local address 127.8.64.190:43183)
0924 01:01:39.958533 (+ 1129us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:39.958548 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:39.958973 (+ 425us) server_negotiation.cc:407] Connection header received
0924 01:01:39.961389 (+ 2416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:39.961400 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:39.961731 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:39.961941 (+ 210us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:39.963145 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:39.965131 (+ 1986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:39.966862 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:39.967876 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:39.968069 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:39.971922 (+ 3853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:39.971931 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:39.971943 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:01:39.971972 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:39.975197 (+ 3225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:39.976153 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:39.976163 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:39.976165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:39.976326 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:39.976872 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:39.976883 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:39.976885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:39.977396 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:39.977561 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:39.978220 (+ 659us) server_negotiation.cc:299] Negotiation successful
0924 01:01:39.978656 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":244,"server-negotiator.queue_time_us":759,"thread_start_us":475,"threads_started":1}
I20260924 01:01:39.979863 30955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:39.957405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:43183 (local address 127.0.0.1:53604)
0924 01:01:39.958784 (+ 1379us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:39.958837 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0924 01:01:39.961003 (+ 2166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:01:39.961974 (+ 971us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:01:39.961979 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:01:39.962605 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0924 01:01:39.962930 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:39.962946 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:39.965335 (+ 2389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:39.965339 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:39.966701 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:39.966713 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:39.968110 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:39.968115 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:39.968266 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:39.969053 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 01:01:39.969075 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0924 01:01:39.971668 (+ 2593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 01:01:39.975337 (+ 3669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:01:39.975351 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:01:39.975365 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:01:39.975977 (+ 612us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:01:39.976476 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:01:39.976481 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:01:39.976485 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:01:39.976695 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:01:39.977541 (+ 846us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 01:01:39.977547 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 01:01:39.977736 (+ 189us) client_negotiation.cc:769] Sending connection context
0924 01:01:39.978008 (+ 272us) client_negotiation.cc:240] Negotiation successful
0924 01:01:39.978538 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1103,"mutex_wait_us":2,"spinlock_wait_cycles":14720,"thread_start_us":838,"threads_started":1}
I20260924 01:01:39.983421 30952 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43859 } has no permanent_uuid. Determining permanent_uuid...
I20260924 01:01:39.984768 30955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:39.984241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:43859 (local address 127.0.0.1:52866)
0924 01:01:39.984452 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:39.984603 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.189:43859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260924 01:01:39.985077 30902 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.64.189:43859: connect: Connection refused (error 111)
I20260924 01:01:39.986778 30955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:39.986269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:43859 (local address 127.0.0.1:52880)
0924 01:01:39.986466 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:39.986567 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.189:43859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260924 01:01:39.986977 30952 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.64.189:43859: Network error: Client connection negotiation failed: client connection to 127.8.64.189:43859: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.189.keytab.
Entry for principal kudu/127.8.64.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.189.keytab.
I20260924 01:01:40.013862 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.189:43859
--webserver_interface=127.8.64.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.189.keytab
--principal=kudu/127.8.64.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:40473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:43183,127.8.64.189:43859,127.8.64.188:36541
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 01:01:40.036331 30952 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43859 } attempt: 1
I20260924 01:01:40.037681 30955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:40.037062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:43859 (local address 127.0.0.1:52884)
0924 01:01:40.037309 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:40.037474 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.189:43859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260924 01:01:40.038905 30955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:40.038381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:43859 (local address 127.0.0.1:52886)
0924 01:01:40.038572 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:40.038674 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.189:43859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260924 01:01:40.039113 30952 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.64.189:43859: Network error: Client connection negotiation failed: client connection to 127.8.64.189:43859: connect: Connection refused (error 111)
I20260924 01:01:40.107395 30952 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43859 } attempt: 2
I20260924 01:01:40.108717 30955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:40.108154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:43859 (local address 127.0.0.1:52890)
0924 01:01:40.108391 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:40.108523 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.189:43859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260924 01:01:40.109833 30955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:40.109394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:43859 (local address 127.0.0.1:52898)
0924 01:01:40.109586 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:40.109673 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.189:43859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260924 01:01:40.110054 30952 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.64.189:43859: Network error: Client connection negotiation failed: client connection to 127.8.64.189:43859: connect: Connection refused (error 111)
I20260924 01:01:40.201327 30952 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43859 } attempt: 3
I20260924 01:01:40.202618 30955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:40.202060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:43859 (local address 127.0.0.1:52908)
0924 01:01:40.202320 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:40.202446 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.189:43859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260924 01:01:40.203755 30955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:40.203320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:43859 (local address 127.0.0.1:52916)
0924 01:01:40.203507 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:40.203596 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.189:43859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260924 01:01:40.203946 30952 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.64.189:43859: Network error: Client connection negotiation failed: client connection to 127.8.64.189:43859: connect: Connection refused (error 111)
W20260924 01:01:40.346207 30959 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 01:01:40.346568 30959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:01:40.346695 30959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:01:40.346812 30959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260924 01:01:40.347199 30952 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43859 } attempt: 4
I20260924 01:01:40.348429 30955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:40.347907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:43859 (local address 127.0.0.1:52932)
0924 01:01:40.348212 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:40.348300 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.189:43859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182}
I20260924 01:01:40.349465 30955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:40.349139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:43859 (local address 127.0.0.1:52944)
0924 01:01:40.349259 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:40.349339 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.189:43859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260924 01:01:40.349697 30952 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.64.189:43859: Network error: Client connection negotiation failed: client connection to 127.8.64.189:43859: connect: Connection refused (error 111)
W20260924 01:01:40.354844 30959 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 01:01:40.354923 30959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:01:40.355042 30959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:01:40.355115 30959 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 01:01:40.355222 30959 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 01:01:40.364910 30959 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:40473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:43183,127.8.64.189:43859,127.8.64.188:36541
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.189.keytab
--principal=kudu/127.8.64.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.189:43859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.64.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:01:40.366155 30959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:01:40.367486 30959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:01:40.379602 30964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:40.379666 30965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:40.379642 30967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:01:40.380414 30959 server_base.cc:1034] running on GCE node
I20260924 01:01:40.381156 30959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:01:40.382526 30959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:01:40.383762 30959 hybrid_clock.cc:648] HybridClock initialized: now 1790211700383698 us; error 39 us; skew 500 ppm
I20260924 01:01:40.384083 30959 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 01:01:40 dist-test-slave-mmhs krb5kdc[30872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211700, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 01:01:40.387253 30959 init.cc:377] Logged in from keytab as kudu/127.8.64.189@KRBTEST.COM (short username kudu)
I20260924 01:01:40.388746 30959 webserver.cc:460] Webserver started at http://127.8.64.189:45213/ using document root <none> and password file <none>
I20260924 01:01:40.389432 30959 fs_manager.cc:359] Metadata directory not provided
I20260924 01:01:40.389528 30959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:01:40.389788 30959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:01:40.392728 30959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-1/data/instance:
uuid: "924b5ef69679445abd70279b32ec0554"
format_stamp: "Formatted at 2026-09-24 01:01:40 on dist-test-slave-mmhs"
server_key: "41a302a072fa01823f4e8eb122f80fa2"
server_key_iv: "d39b3288680153b36edfbc547b14baf0"
server_key_version: "encryptionkey@0"
I20260924 01:01:40.393451 30959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-1/wal/instance:
uuid: "924b5ef69679445abd70279b32ec0554"
format_stamp: "Formatted at 2026-09-24 01:01:40 on dist-test-slave-mmhs"
server_key: "41a302a072fa01823f4e8eb122f80fa2"
server_key_iv: "d39b3288680153b36edfbc547b14baf0"
server_key_version: "encryptionkey@0"
I20260924 01:01:40.398464 30959 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260924 01:01:40.402046 30974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:01:40.403546 30959 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 01:01:40.403712 30959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-1/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-1/wal
uuid: "924b5ef69679445abd70279b32ec0554"
format_stamp: "Formatted at 2026-09-24 01:01:40 on dist-test-slave-mmhs"
server_key: "41a302a072fa01823f4e8eb122f80fa2"
server_key_iv: "d39b3288680153b36edfbc547b14baf0"
server_key_version: "encryptionkey@0"
I20260924 01:01:40.403930 30959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:01:40.420697 30959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:01:40.424898 30959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:01:40.425166 30959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:01:40.444837 30959 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.189:43859
I20260924 01:01:40.444888 31025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.189:43859 every 8 connection(s)
I20260924 01:01:40.446245 30959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-1/data/info.pb
I20260924 01:01:40.450011 31026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:01:40.450145 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 30959
I20260924 01:01:40.450361 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-1/wal/instance
I20260924 01:01:40.450800 8450 external_mini_cluster.cc:1428] Setting key 6b89288a58d02ba81564a49b08d22588
I20260924 01:01:40.456569 31026 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.64.190" port: 43183 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 01:01:40 dist-test-slave-mmhs krb5kdc[30872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211700, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.189@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.64.188@KRBTEST.COM; defaulting to no policy
I20260924 01:01:40.478649 30954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:40.459107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53610 (local address 127.8.64.190:43183)
0924 01:01:40.459275 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:40.459279 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:40.460801 (+ 1522us) server_negotiation.cc:407] Connection header received
0924 01:01:40.462619 (+ 1818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:40.462628 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:40.462734 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:40.462869 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:40.464294 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:40.465434 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:40.467090 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:40.467852 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:40.467989 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:40.471674 (+ 3685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:40.471682 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:40.471690 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:01:40.471736 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:40.475565 (+ 3829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:40.476310 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:40.476316 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:40.476321 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:40.476482 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:40.476879 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:40.476887 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:40.476893 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:40.477293 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:40.477442 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:40.478179 (+ 737us) server_negotiation.cc:299] Negotiation successful
0924 01:01:40.478402 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.188@KRBTEST.COM" created.
I20260924 01:01:40.479959 31029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:40.459695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:43183 (local address 127.0.0.1:53610)
0924 01:01:40.460550 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:40.460593 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 01:01:40.462343 (+ 1750us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:01:40.463052 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:01:40.463059 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:01:40.463581 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0924 01:01:40.464121 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:40.464138 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:40.465624 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:40.465628 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:40.466935 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:40.466947 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:40.468072 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:40.468077 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:40.468289 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:40.469024 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 01:01:40.469045 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0924 01:01:40.471448 (+ 2403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 01:01:40.475716 (+ 4268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:01:40.475722 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:01:40.475736 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:01:40.476171 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:01:40.476603 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:01:40.476606 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:01:40.476608 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:01:40.476764 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:01:40.477448 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 01:01:40.477464 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 01:01:40.477833 (+ 369us) client_negotiation.cc:769] Sending connection context
0924 01:01:40.478061 (+ 228us) client_negotiation.cc:240] Negotiation successful
0924 01:01:40.478564 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":368,"threads_started":1}
I20260924 01:01:40.481685 31026 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43859 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 01:01:40 dist-test-slave-mmhs krb5kdc[30872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211700, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.189@KRBTEST.COM for kudu/127.8.64.189@KRBTEST.COM
I20260924 01:01:40.498343 31029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:40.482596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:43859 (local address 127.0.0.1:52952)
0924 01:01:40.482810 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:40.482831 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0924 01:01:40.482937 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:01:40.483904 (+ 967us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:01:40.483911 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:01:40.484395 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0924 01:01:40.484602 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:40.484617 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:40.485965 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:40.485973 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:40.487528 (+ 1555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:40.487545 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:40.488359 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:40.488371 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:40.488585 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:40.489195 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 01:01:40.489221 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0924 01:01:40.491593 (+ 2372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 01:01:40.495922 (+ 4329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:01:40.495926 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:01:40.495929 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:01:40.496284 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:01:40.496727 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:01:40.496730 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:01:40.496732 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:01:40.496858 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:01:40.497608 (+ 750us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 01:01:40.497611 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 01:01:40.497719 (+ 108us) client_negotiation.cc:769] Sending connection context
0924 01:01:40.497820 (+ 101us) client_negotiation.cc:240] Negotiation successful
0924 01:01:40.498028 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260924 01:01:40.498822 31030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:40.482714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52952 (local address 127.8.64.189:43859)
0924 01:01:40.483424 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:40.483430 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:40.483455 (+ 25us) server_negotiation.cc:407] Connection header received
0924 01:01:40.483497 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:40.483559 (+ 62us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:40.483739 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:40.483852 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:40.484771 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:40.485826 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:40.487663 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:40.488209 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:40.488310 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:40.491802 (+ 3492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:40.491817 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:40.491838 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:01:40.491885 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:40.495786 (+ 3901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:40.496435 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:40.496446 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:40.496452 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:40.496622 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:40.496965 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:40.496972 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:40.496978 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:40.497452 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:40.497709 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:40.498030 (+ 321us) server_negotiation.cc:299] Negotiation successful
0924 01:01:40.498333 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":431,"threads_started":1}
I20260924 01:01:40.501183 31026 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36541 } has no permanent_uuid. Determining permanent_uuid...
I20260924 01:01:40.502202 31029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:40.501798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:36541 (local address 127.0.0.1:48378)
0924 01:01:40.501959 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:40.502075 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:36541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260924 01:01:40.502446 30977 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.64.188:36541: connect: Connection refused (error 111)
I20260924 01:01:40.503937 31029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:40.503631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:36541 (local address 127.0.0.1:48394)
0924 01:01:40.503773 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:40.503846 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:36541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260924 01:01:40.504182 31026 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.64.188:36541: Network error: Client connection negotiation failed: client connection to 127.8.64.188:36541: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.188.keytab.
Entry for principal kudu/127.8.64.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.188.keytab.
I20260924 01:01:40.517809 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.188:36541
--webserver_interface=127.8.64.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.188.keytab
--principal=kudu/127.8.64.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:40473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:43183,127.8.64.189:43859,127.8.64.188:36541
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 01:01:40.553539 31026 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36541 } attempt: 1
I20260924 01:01:40.554867 31029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:40.554342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:36541 (local address 127.0.0.1:48402)
0924 01:01:40.554561 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:40.554694 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:36541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260924 01:01:40.555939 31029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:40.555561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:36541 (local address 127.0.0.1:48418)
0924 01:01:40.555713 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:40.555828 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:36541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260924 01:01:40.556164 31026 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.64.188:36541: Network error: Client connection negotiation failed: client connection to 127.8.64.188:36541: connect: Connection refused (error 111)
I20260924 01:01:40.624430 31026 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36541 } attempt: 2
I20260924 01:01:40.625566 31029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:40.625091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:36541 (local address 127.0.0.1:48422)
0924 01:01:40.625300 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:40.625404 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:36541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260924 01:01:40.626605 31029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:40.626222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:36541 (local address 127.0.0.1:48428)
0924 01:01:40.626404 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:40.626491 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:36541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260924 01:01:40.626825 31026 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.64.188:36541: Network error: Client connection negotiation failed: client connection to 127.8.64.188:36541: connect: Connection refused (error 111)
I20260924 01:01:40.648976 30952 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43859 } attempt: 5
Sep 24 01:01:40 dist-test-slave-mmhs krb5kdc[30872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211699, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for kudu/127.8.64.189@KRBTEST.COM
I20260924 01:01:40.663970 31030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:40.649871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52964 (local address 127.8.64.189:43859)
0924 01:01:40.650058 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:40.650063 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:40.650107 (+ 44us) server_negotiation.cc:407] Connection header received
0924 01:01:40.650388 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:40.650393 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:40.650479 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:40.650569 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:40.651594 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:40.652388 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:40.654069 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:40.654609 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:40.654715 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:40.658508 (+ 3793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:40.658512 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:40.658516 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:01:40.658546 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:40.661137 (+ 2591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:40.661890 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:40.661898 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:40.661904 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:40.662088 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:40.662538 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:40.662545 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:40.662564 (+ 19us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:40.662848 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:40.662996 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:40.663421 (+ 425us) server_negotiation.cc:299] Negotiation successful
0924 01:01:40.663637 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260924 01:01:40.663967 30955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:40.649736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:43859 (local address 127.0.0.1:52964)
0924 01:01:40.649999 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:40.650037 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0924 01:01:40.650205 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:01:40.650643 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:01:40.650648 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:01:40.651181 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0924 01:01:40.651423 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:40.651435 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:40.652525 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:40.652531 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:40.653904 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:40.653920 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:40.654772 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:40.654778 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:40.655017 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:40.655743 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 01:01:40.655767 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0924 01:01:40.658271 (+ 2504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 01:01:40.661264 (+ 2993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:01:40.661271 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:01:40.661275 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:01:40.661787 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:01:40.662223 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:01:40.662230 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:01:40.662234 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:01:40.662443 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:01:40.662973 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 01:01:40.662978 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 01:01:40.663139 (+ 161us) client_negotiation.cc:769] Sending connection context
0924 01:01:40.663324 (+ 185us) client_negotiation.cc:240] Negotiation successful
0924 01:01:40.663621 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260924 01:01:40.664986 30952 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36541 } has no permanent_uuid. Determining permanent_uuid...
I20260924 01:01:40.665939 30955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:40.665591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:36541 (local address 127.0.0.1:48444)
0924 01:01:40.665739 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:40.665808 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:36541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260924 01:01:40.666801 30955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:40.666494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:36541 (local address 127.0.0.1:48460)
0924 01:01:40.666652 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:40.666717 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:36541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260924 01:01:40.667009 30952 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.64.188:36541: Network error: Client connection negotiation failed: client connection to 127.8.64.188:36541: connect: Connection refused (error 111)
I20260924 01:01:40.718042 31026 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36541 } attempt: 3
I20260924 01:01:40.718236 30952 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36541 } attempt: 1
I20260924 01:01:40.719275 30955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:40.718804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:36541 (local address 127.0.0.1:48474)
0924 01:01:40.719018 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:40.719110 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:36541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260924 01:01:40.719434 31029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:40.718824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:36541 (local address 127.0.0.1:48468)
0924 01:01:40.719144 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:40.719245 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:36541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186}
I20260924 01:01:40.720448 30955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:40.720058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:36541 (local address 127.0.0.1:48486)
0924 01:01:40.720234 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:40.720321 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:36541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260924 01:01:40.720568 31029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:40.720114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:36541 (local address 127.0.0.1:48498)
0924 01:01:40.720344 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:40.720424 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:36541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134}
W20260924 01:01:40.720672 30952 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.64.188:36541: Network error: Client connection negotiation failed: client connection to 127.8.64.188:36541: connect: Connection refused (error 111)
W20260924 01:01:40.720839 31026 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.64.188:36541: Network error: Client connection negotiation failed: client connection to 127.8.64.188:36541: connect: Connection refused (error 111)
I20260924 01:01:40.788906 30952 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36541 } attempt: 2
I20260924 01:01:40.790251 30955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:40.789693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:36541 (local address 127.0.0.1:48508)
0924 01:01:40.789968 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:40.790072 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:36541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260924 01:01:40.791471 30955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:40.791044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:36541 (local address 127.0.0.1:48512)
0924 01:01:40.791215 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:40.791305 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:36541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260924 01:01:40.791754 30952 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.64.188:36541: Network error: Client connection negotiation failed: client connection to 127.8.64.188:36541: connect: Connection refused (error 111)
W20260924 01:01:40.853605 31033 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 01:01:40.853940 31033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:01:40.854061 31033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:01:40.854192 31033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:01:40.861414 31033 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 01:01:40.861498 31033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:01:40.861603 31033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:01:40.861687 31033 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 01:01:40.861799 31033 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 01:01:40.864058 31026 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36541 } attempt: 4
I20260924 01:01:40.865278 31029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:40.864793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:36541 (local address 127.0.0.1:48522)
0924 01:01:40.865031 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:40.865122 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:36541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260924 01:01:40.866657 31029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:40.866126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:36541 (local address 127.0.0.1:48528)
0924 01:01:40.866339 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:40.866482 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:36541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260924 01:01:40.866992 31026 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.64.188:36541: Network error: Client connection negotiation failed: client connection to 127.8.64.188:36541: connect: Connection refused (error 111)
I20260924 01:01:40.872637 31033 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:40473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:43183,127.8.64.189:43859,127.8.64.188:36541
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.188.keytab
--principal=kudu/127.8.64.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.188:36541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.64.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:01:40.873961 31033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:01:40.875363 31033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:01:40.887214 31039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:40.887269 31038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:40.887274 31041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:01:40.888252 31033 server_base.cc:1034] running on GCE node
I20260924 01:01:40.889091 31033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:01:40.890550 31033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:01:40.891779 31033 hybrid_clock.cc:648] HybridClock initialized: now 1790211700891712 us; error 45 us; skew 500 ppm
I20260924 01:01:40.892122 31033 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 01:01:40 dist-test-slave-mmhs krb5kdc[30872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211700, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 01:01:40.895380 31033 init.cc:377] Logged in from keytab as kudu/127.8.64.188@KRBTEST.COM (short username kudu)
I20260924 01:01:40.896862 31033 webserver.cc:460] Webserver started at http://127.8.64.188:36387/ using document root <none> and password file <none>
I20260924 01:01:40.897537 31033 fs_manager.cc:359] Metadata directory not provided
I20260924 01:01:40.897625 31033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:01:40.898003 31033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:01:40.897979 30952 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36541 } attempt: 3
I20260924 01:01:40.899156 30955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:40.898622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:36541 (local address 127.0.0.1:48538)
0924 01:01:40.898841 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:40.898943 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:36541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260924 01:01:40.900205 30955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:40.899819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:36541 (local address 127.0.0.1:48542)
0924 01:01:40.900030 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:40.900107 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:36541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
W20260924 01:01:40.900475 30952 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.64.188:36541: Network error: Client connection negotiation failed: client connection to 127.8.64.188:36541: connect: Connection refused (error 111)
I20260924 01:01:40.901598 31033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-2/data/instance:
uuid: "09f057fa17c842a1ab4f46fde4258923"
format_stamp: "Formatted at 2026-09-24 01:01:40 on dist-test-slave-mmhs"
server_key: "70b75071a1d0352947aa9c936f8cecf4"
server_key_iv: "0b2768e8c0470af47bb4b83409063353"
server_key_version: "encryptionkey@0"
I20260924 01:01:40.902423 31033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-2/wal/instance:
uuid: "09f057fa17c842a1ab4f46fde4258923"
format_stamp: "Formatted at 2026-09-24 01:01:40 on dist-test-slave-mmhs"
server_key: "70b75071a1d0352947aa9c936f8cecf4"
server_key_iv: "0b2768e8c0470af47bb4b83409063353"
server_key_version: "encryptionkey@0"
I20260924 01:01:40.907652 31033 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260924 01:01:40.911058 31048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:01:40.912402 31033 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260924 01:01:40.912578 31033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-2/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-2/wal
uuid: "09f057fa17c842a1ab4f46fde4258923"
format_stamp: "Formatted at 2026-09-24 01:01:40 on dist-test-slave-mmhs"
server_key: "70b75071a1d0352947aa9c936f8cecf4"
server_key_iv: "0b2768e8c0470af47bb4b83409063353"
server_key_version: "encryptionkey@0"
I20260924 01:01:40.912792 31033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:01:40.948752 31033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:01:40.952946 31033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:01:40.953233 31033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:01:40.971295 31033 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.188:36541
I20260924 01:01:40.971313 31099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.188:36541 every 8 connection(s)
I20260924 01:01:40.972555 31033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-2/data/info.pb
I20260924 01:01:40.974573 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 31033
I20260924 01:01:40.974774 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790211343099518-8450-0/minicluster-data/master-2/wal/instance
I20260924 01:01:40.975271 8450 external_mini_cluster.cc:1428] Setting key 5a9d7a5b8bfa1f036d80b6b945a6c6de
I20260924 01:01:40.976684 31100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:01:40.986315 31100 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.64.190" port: 43183 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 01:01:40 dist-test-slave-mmhs krb5kdc[30872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.188@KRBTEST.COM
Sep 24 01:01:40 dist-test-slave-mmhs krb5kdc[30872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211700, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.188@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 01:01:41.000092 31103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:40.977076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48548 (local address 127.8.64.188:36541)
0924 01:01:40.977806 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:40.977818 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:40.977855 (+ 37us) server_negotiation.cc:407] Connection header received
0924 01:01:40.979406 (+ 1551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:40.979427 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:40.979781 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:40.980101 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:40.982049 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:40.983385 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:40.985364 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:40.986012 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:40.986147 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:40.991620 (+ 5473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:40.991630 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:40.991641 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:01:40.991684 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:40.995568 (+ 3884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:40.996542 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:40.996550 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:40.996559 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:40.996759 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:40.997144 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:40.997151 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:40.997154 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:40.997703 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:40.998086 (+ 383us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:40.998566 (+ 480us) server_negotiation.cc:299] Negotiation successful
0924 01:01:40.999051 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":332,"threads_started":1}
I20260924 01:01:41.005216 31105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:40.989504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:43183 (local address 127.0.0.1:53624)
0924 01:01:40.990512 (+ 1008us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:40.990557 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0924 01:01:40.990701 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:01:40.991139 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:01:40.991144 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:01:40.992000 (+ 856us) client_negotiation.cc:189] Negotiated authn=SASL
0924 01:01:40.992383 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:40.992393 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:40.993712 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:40.993717 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:40.995497 (+ 1780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:40.995513 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:40.996620 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:40.996627 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:40.996810 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:40.997395 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 01:01:40.997414 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0924 01:01:40.999456 (+ 2042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 01:01:41.002387 (+ 2931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:01:41.002397 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:01:41.002403 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:01:41.002937 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:01:41.003417 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:01:41.003425 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:01:41.003428 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:01:41.003613 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:01:41.004150 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 01:01:41.004158 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 01:01:41.004322 (+ 164us) client_negotiation.cc:769] Sending connection context
0924 01:01:41.004489 (+ 167us) client_negotiation.cc:240] Negotiation successful
0924 01:01:41.004788 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":767,"thread_start_us":622,"threads_started":1}
I20260924 01:01:41.005254 31104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:40.989531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53624 (local address 127.8.64.190:43183)
0924 01:01:40.990115 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:40.990122 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:40.990655 (+ 533us) server_negotiation.cc:407] Connection header received
0924 01:01:40.990845 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:40.990854 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:40.990957 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:40.991081 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:40.992525 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:40.993423 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:40.995677 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:40.996438 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:40.996576 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:40.999640 (+ 3064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:40.999645 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:40.999648 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:01:40.999681 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:41.002181 (+ 2500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:41.003088 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:41.003095 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:41.003101 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:41.003304 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:41.003729 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:41.003732 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:41.003734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:41.003982 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:41.004199 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:41.004605 (+ 406us) server_negotiation.cc:299] Negotiation successful
0924 01:01:41.004807 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":292,"threads_started":1}
I20260924 01:01:41.006531 31100 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43859 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 01:01:41 dist-test-slave-mmhs krb5kdc[30872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
Sep 24 01:01:41 dist-test-slave-mmhs krb5kdc[30872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211700, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.188@KRBTEST.COM for kudu/127.8.64.189@KRBTEST.COM
I20260924 01:01:41.020120 31105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:41.007237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:43859 (local address 127.0.0.1:52966)
0924 01:01:41.007377 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:41.007395 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0924 01:01:41.007486 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:01:41.007792 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:01:41.007797 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:01:41.008118 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0924 01:01:41.008239 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:41.008246 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:41.009209 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:41.009217 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:41.010070 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:41.010080 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:41.010844 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:41.010849 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:41.011027 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:41.011510 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 01:01:41.011525 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0924 01:01:41.013696 (+ 2171us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 01:01:41.017535 (+ 3839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:01:41.017540 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:01:41.017544 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:01:41.018047 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:01:41.018440 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:01:41.018448 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:01:41.018454 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:01:41.018633 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:01:41.019184 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 01:01:41.019188 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 01:01:41.019329 (+ 141us) client_negotiation.cc:769] Sending connection context
0924 01:01:41.019478 (+ 149us) client_negotiation.cc:240] Negotiation successful
0924 01:01:41.019747 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260924 01:01:41.020172 31030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:41.007262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52966 (local address 127.8.64.189:43859)
0924 01:01:41.007453 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:41.007459 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:41.007480 (+ 21us) server_negotiation.cc:407] Connection header received
0924 01:01:41.007571 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:41.007577 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:41.007654 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:41.007780 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:41.008375 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:41.009077 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:41.010207 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:41.010694 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:41.010819 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:41.013882 (+ 3063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:41.013891 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:41.013897 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:01:41.013954 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:41.017395 (+ 3441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:41.018154 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:41.018159 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:41.018163 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:41.018337 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:41.018744 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:41.018750 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:41.018752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:41.019056 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:41.019189 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:41.019579 (+ 390us) server_negotiation.cc:299] Negotiation successful
0924 01:01:41.019784 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260924 01:01:41.020210 31106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:41.002970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53630 (local address 127.8.64.190:43183)
0924 01:01:41.003608 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:41.003615 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:41.003641 (+ 26us) server_negotiation.cc:407] Connection header received
0924 01:01:41.003696 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:41.003702 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:41.003776 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:41.003897 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:41.005198 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:41.006421 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:41.008222 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:41.008774 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:41.008925 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:41.013598 (+ 4673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:41.013605 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:41.013610 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:01:41.013668 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:41.017208 (+ 3540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:41.018035 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:41.018039 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:41.018042 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:41.018195 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:41.018650 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:41.018653 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:41.018655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:41.018999 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:41.019255 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:41.019579 (+ 324us) server_negotiation.cc:299] Negotiation successful
0924 01:01:41.019789 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":292,"threads_started":1}
I20260924 01:01:41.021366 31100 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36541 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 01:01:41 dist-test-slave-mmhs krb5kdc[30872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211700, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.188@KRBTEST.COM for kudu/127.8.64.188@KRBTEST.COM
Sep 24 01:01:41 dist-test-slave-mmhs krb5kdc[30872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.189@KRBTEST.COM
I20260924 01:01:41.035727 31103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:41.022139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48558 (local address 127.8.64.188:36541)
0924 01:01:41.022277 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:41.022283 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:41.022322 (+ 39us) server_negotiation.cc:407] Connection header received
0924 01:01:41.022381 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:41.022388 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:41.022457 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:41.022546 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:41.023242 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:41.024303 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:41.025704 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:41.026373 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:41.026494 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:41.029767 (+ 3273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:41.029792 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:41.029801 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:01:41.029848 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:41.032869 (+ 3021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:41.033805 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:41.033810 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:41.033812 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:41.033948 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:41.034414 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:41.034418 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:41.034420 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:41.034705 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:41.034832 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:41.035232 (+ 400us) server_negotiation.cc:299] Negotiation successful
0924 01:01:41.035455 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":44}
I20260924 01:01:41.036084 31105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:41.021998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:36541 (local address 127.0.0.1:48558)
0924 01:01:41.022104 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:41.022117 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 01:01:41.022188 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:01:41.022581 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:01:41.022586 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:01:41.022985 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0924 01:01:41.023106 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:41.023114 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:41.024440 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:41.024445 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:41.025580 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:41.025590 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:41.026551 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:41.026556 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:41.026773 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:41.027287 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 01:01:41.027310 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0924 01:01:41.029575 (+ 2265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 01:01:41.033103 (+ 3528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:01:41.033126 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:01:41.033132 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:01:41.033668 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:01:41.034109 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:01:41.034112 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:01:41.034115 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:01:41.034295 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:01:41.034843 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 01:01:41.034848 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 01:01:41.034999 (+ 151us) client_negotiation.cc:769] Sending connection context
0924 01:01:41.035159 (+ 160us) client_negotiation.cc:240] Negotiation successful
0924 01:01:41.035643 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":113}
I20260924 01:01:41.038996 31030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:41.021771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52976 (local address 127.8.64.189:43859)
0924 01:01:41.021937 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:41.021943 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:41.022408 (+ 465us) server_negotiation.cc:407] Connection header received
0924 01:01:41.022578 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:41.022583 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:41.022652 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:41.022753 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:41.024133 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:41.025155 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:41.026725 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:41.027356 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:41.027490 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:41.032423 (+ 4933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:41.032427 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:41.032431 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:01:41.032460 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:41.035774 (+ 3314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:41.036541 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:41.036547 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:41.036551 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:41.036727 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:41.037412 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:41.037419 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:41.037422 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:41.037822 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:41.038039 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:41.038477 (+ 438us) server_negotiation.cc:299] Negotiation successful
0924 01:01:41.038651 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260924 01:01:41.041929 31100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09f057fa17c842a1ab4f46fde4258923: Bootstrap starting.
I20260924 01:01:41.044077 8450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 01:01:41.045574 31100 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09f057fa17c842a1ab4f46fde4258923: Neither blocks nor log segments found. Creating new log.
I20260924 01:01:41.046826 31100 log.cc:824] T 00000000000000000000000000000000 P 09f057fa17c842a1ab4f46fde4258923: Log is configured to *not* fsync() on all Append() calls
I20260924 01:01:41.049947 31100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09f057fa17c842a1ab4f46fde4258923: No bootstrap required, opened a new log
I20260924 01:01:41.046854 30927 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "b90c9c5c753d4c8ab5e279460aaa7c06"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53630
I20260924 01:01:41.060729 31100 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09f057fa17c842a1ab4f46fde4258923 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90c9c5c753d4c8ab5e279460aaa7c06" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 43183 } } peers { permanent_uuid: "924b5ef69679445abd70279b32ec0554" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43859 } } peers { permanent_uuid: "09f057fa17c842a1ab4f46fde4258923" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36541 } }
I20260924 01:01:41.061158 31100 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09f057fa17c842a1ab4f46fde4258923 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09f057fa17c842a1ab4f46fde4258923, State: Initialized, Role: FOLLOWER
I20260924 01:01:41.061659 31100 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09f057fa17c842a1ab4f46fde4258923 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90c9c5c753d4c8ab5e279460aaa7c06" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 43183 } } peers { permanent_uuid: "924b5ef69679445abd70279b32ec0554" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43859 } } peers { permanent_uuid: "09f057fa17c842a1ab4f46fde4258923" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36541 } }
I20260924 01:01:41.062673 31108 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09f057fa17c842a1ab4f46fde4258923 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90c9c5c753d4c8ab5e279460aaa7c06" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 43183 } } peers { permanent_uuid: "924b5ef69679445abd70279b32ec0554" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43859 } } peers { permanent_uuid: "09f057fa17c842a1ab4f46fde4258923" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36541 } } }
I20260924 01:01:41.063035 31108 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09f057fa17c842a1ab4f46fde4258923 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 01:01:41.063844 31100 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09f057fa17c842a1ab4f46fde4258923 [sys.catalog]: configured and running, proceeding with master startup.
W20260924 01:01:41.073380 31119 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 09f057fa17c842a1ab4f46fde4258923: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 01:01:41.073523 31119 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 01:01:41.077631 30952 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36541 } attempt: 4
I20260924 01:01:41.071152 31033 master_runner.cc:186] Error getting master registration for 127.8.64.190:43183: OK, instance_id { permanent_uuid: "b90c9c5c753d4c8ab5e279460aaa7c06" instance_seqno: 1790211699936767 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 01:01:41.082537 31033 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Sep 24 01:01:41 dist-test-slave-mmhs krb5kdc[30872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211699, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for kudu/127.8.64.188@KRBTEST.COM
I20260924 01:01:41.093315 30955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:41.078236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:36541 (local address 127.0.0.1:48566)
0924 01:01:41.078439 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:41.078461 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0924 01:01:41.078579 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:01:41.079113 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:01:41.079121 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:01:41.079583 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0924 01:01:41.079736 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:41.079745 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:41.081176 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:41.081182 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:41.082750 (+ 1568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:41.082767 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:41.083535 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:41.083542 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:41.083728 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:41.084341 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 01:01:41.084363 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0924 01:01:41.086471 (+ 2108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 01:01:41.090529 (+ 4058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:01:41.090535 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:01:41.090539 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:01:41.091030 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:01:41.091446 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:01:41.091451 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:01:41.091455 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:01:41.091622 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:01:41.092316 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 01:01:41.092321 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 01:01:41.092501 (+ 180us) client_negotiation.cc:769] Sending connection context
0924 01:01:41.092634 (+ 133us) client_negotiation.cc:240] Negotiation successful
0924 01:01:41.092918 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76}
I20260924 01:01:41.093382 31103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:41.078469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48566 (local address 127.8.64.188:36541)
0924 01:01:41.078814 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:41.078821 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:41.078837 (+ 16us) server_negotiation.cc:407] Connection header received
0924 01:01:41.078902 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:41.078908 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:41.078979 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:41.079072 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:41.079892 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:41.081046 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:41.082898 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:41.083368 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:41.083514 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:41.086671 (+ 3157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:41.086678 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:41.086683 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:01:41.086720 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:41.090394 (+ 3674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:41.091170 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:41.091176 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:41.091179 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:41.091325 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:41.091738 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:41.091743 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:41.091747 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:41.092200 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:41.092377 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:41.092791 (+ 414us) server_negotiation.cc:299] Negotiation successful
0924 01:01:41.093014 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204}
I20260924 01:01:41.099469 30952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b90c9c5c753d4c8ab5e279460aaa7c06: Bootstrap starting.
I20260924 01:01:41.103468 30952 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b90c9c5c753d4c8ab5e279460aaa7c06: Neither blocks nor log segments found. Creating new log.
I20260924 01:01:41.104754 30952 log.cc:824] T 00000000000000000000000000000000 P b90c9c5c753d4c8ab5e279460aaa7c06: Log is configured to *not* fsync() on all Append() calls
I20260924 01:01:41.108207 30952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b90c9c5c753d4c8ab5e279460aaa7c06: No bootstrap required, opened a new log
I20260924 01:01:41.113260 30952 raft_consensus.cc:348] T 00000000000000000000000000000000 P b90c9c5c753d4c8ab5e279460aaa7c06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90c9c5c753d4c8ab5e279460aaa7c06" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 43183 } } peers { permanent_uuid: "924b5ef69679445abd70279b32ec0554" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43859 } } peers { permanent_uuid: "09f057fa17c842a1ab4f46fde4258923" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36541 } }
I20260924 01:01:41.113586 30952 raft_consensus.cc:729] T 00000000000000000000000000000000 P b90c9c5c753d4c8ab5e279460aaa7c06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b90c9c5c753d4c8ab5e279460aaa7c06, State: Initialized, Role: FOLLOWER
I20260924 01:01:41.114007 30952 consensus_queue.cc:260] T 00000000000000000000000000000000 P b90c9c5c753d4c8ab5e279460aaa7c06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90c9c5c753d4c8ab5e279460aaa7c06" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 43183 } } peers { permanent_uuid: "924b5ef69679445abd70279b32ec0554" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43859 } } peers { permanent_uuid: "09f057fa17c842a1ab4f46fde4258923" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36541 } }
I20260924 01:01:41.114923 31122 sys_catalog.cc:455] T 00000000000000000000000000000000 P b90c9c5c753d4c8ab5e279460aaa7c06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90c9c5c753d4c8ab5e279460aaa7c06" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 43183 } } peers { permanent_uuid: "924b5ef69679445abd70279b32ec0554" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43859 } } peers { permanent_uuid: "09f057fa17c842a1ab4f46fde4258923" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36541 } } }
I20260924 01:01:41.115305 31122 sys_catalog.cc:458] T 00000000000000000000000000000000 P b90c9c5c753d4c8ab5e279460aaa7c06 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 01:01:41.116057 30952 sys_catalog.cc:564] T 00000000000000000000000000000000 P b90c9c5c753d4c8ab5e279460aaa7c06 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 01:01:41.116144 30927 raft_consensus.cc:482] T 00000000000000000000000000000000 P b90c9c5c753d4c8ab5e279460aaa7c06 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 01:01:41.116354 30927 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b90c9c5c753d4c8ab5e279460aaa7c06 [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:01:41.120203 30927 raft_consensus.cc:504] T 00000000000000000000000000000000 P b90c9c5c753d4c8ab5e279460aaa7c06 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90c9c5c753d4c8ab5e279460aaa7c06" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 43183 } } peers { permanent_uuid: "924b5ef69679445abd70279b32ec0554" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43859 } } peers { permanent_uuid: "09f057fa17c842a1ab4f46fde4258923" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36541 } }
I20260924 01:01:41.122632 31075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b90c9c5c753d4c8ab5e279460aaa7c06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "09f057fa17c842a1ab4f46fde4258923"
I20260924 01:01:41.122892 30927 leader_election.cc:288] T 00000000000000000000000000000000 P b90c9c5c753d4c8ab5e279460aaa7c06 [CANDIDATE]: Term 1 election: Requested vote from peers 924b5ef69679445abd70279b32ec0554 (127.8.64.189:43859), 09f057fa17c842a1ab4f46fde4258923 (127.8.64.188:36541)
I20260924 01:01:41.123131 31075 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09f057fa17c842a1ab4f46fde4258923 [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:01:41.126603 31075 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 09f057fa17c842a1ab4f46fde4258923 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b90c9c5c753d4c8ab5e279460aaa7c06 in term 1.
I20260924 01:01:41.127231 30903 leader_election.cc:302] T 00000000000000000000000000000000 P b90c9c5c753d4c8ab5e279460aaa7c06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09f057fa17c842a1ab4f46fde4258923, b90c9c5c753d4c8ab5e279460aaa7c06; no voters:
I20260924 01:01:41.127702 31122 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b90c9c5c753d4c8ab5e279460aaa7c06 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 01:01:41.137712 31122 raft_consensus.cc:686] T 00000000000000000000000000000000 P b90c9c5c753d4c8ab5e279460aaa7c06 [term 1 LEADER]: Becoming Leader. State: Replica: b90c9c5c753d4c8ab5e279460aaa7c06, State: Running, Role: LEADER
I20260924 01:01:41.138329 31122 consensus_queue.cc:237] T 00000000000000000000000000000000 P b90c9c5c753d4c8ab5e279460aaa7c06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90c9c5c753d4c8ab5e279460aaa7c06" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 43183 } } peers { permanent_uuid: "924b5ef69679445abd70279b32ec0554" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43859 } } peers { permanent_uuid: "09f057fa17c842a1ab4f46fde4258923" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36541 } }
I20260924 01:01:41.142876 31134 sys_catalog.cc:455] T 00000000000000000000000000000000 P b90c9c5c753d4c8ab5e279460aaa7c06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b90c9c5c753d4c8ab5e279460aaa7c06. Latest consensus state: current_term: 1 leader_uuid: "b90c9c5c753d4c8ab5e279460aaa7c06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90c9c5c753d4c8ab5e279460aaa7c06" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 43183 } } peers { permanent_uuid: "924b5ef69679445abd70279b32ec0554" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43859 } } peers { permanent_uuid: "09f057fa17c842a1ab4f46fde4258923" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36541 } } }
I20260924 01:01:41.143301 31134 sys_catalog.cc:458] T 00000000000000000000000000000000 P b90c9c5c753d4c8ab5e279460aaa7c06 [sys.catalog]: This master's current role is: LEADER
I20260924 01:01:41.150180 31137 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 01:01:41.166224 31026 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36541 } attempt: 5
I20260924 01:01:41.167418 31137 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 24 01:01:41 dist-test-slave-mmhs krb5kdc[30872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211700, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.189@KRBTEST.COM for kudu/127.8.64.188@KRBTEST.COM
I20260924 01:01:41.184221 31029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:41.166954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:36541 (local address 127.0.0.1:48576)
0924 01:01:41.167398 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:41.167416 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0924 01:01:41.167533 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:01:41.168471 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:01:41.168476 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:01:41.168986 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0924 01:01:41.169190 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:41.169203 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:41.170740 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:41.170745 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:41.172151 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:41.172165 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:41.173338 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:41.173344 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:41.173551 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:41.174221 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 01:01:41.174244 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0924 01:01:41.177214 (+ 2970us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 01:01:41.181365 (+ 4151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:01:41.181371 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:01:41.181375 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:01:41.181903 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:01:41.182368 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:01:41.182373 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:01:41.182377 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:01:41.182558 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:01:41.183220 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 01:01:41.183225 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 01:01:41.183374 (+ 149us) client_negotiation.cc:769] Sending connection context
0924 01:01:41.183541 (+ 167us) client_negotiation.cc:240] Negotiation successful
0924 01:01:41.183826 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265}
I20260924 01:01:41.185163 31103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:41.167813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48576 (local address 127.8.64.188:36541)
0924 01:01:41.167992 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:41.167998 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:41.168021 (+ 23us) server_negotiation.cc:407] Connection header received
0924 01:01:41.168081 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:41.168086 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:41.168171 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:41.168286 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:41.169370 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:41.170572 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:41.172322 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:41.173021 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:41.173144 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:41.177431 (+ 4287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:41.177437 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:41.177443 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:01:41.177481 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:41.181206 (+ 3725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:41.182046 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:41.182052 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:41.182056 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:41.182210 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:41.182685 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:41.182690 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:41.182694 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:41.183019 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:41.183159 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:41.184534 (+ 1375us) server_negotiation.cc:299] Negotiation successful
0924 01:01:41.184778 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260924 01:01:41.175566 30906 tablet.cc:2369] T 00000000000000000000000000000000 P b90c9c5c753d4c8ab5e279460aaa7c06: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 01:01:41.199170 31026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 924b5ef69679445abd70279b32ec0554: Bootstrap starting.
I20260924 01:01:41.203205 31026 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 924b5ef69679445abd70279b32ec0554: Neither blocks nor log segments found. Creating new log.
I20260924 01:01:41.204488 31026 log.cc:824] T 00000000000000000000000000000000 P 924b5ef69679445abd70279b32ec0554: Log is configured to *not* fsync() on all Append() calls
W20260924 01:01:41.205315 31054 tablet.cc:2369] T 00000000000000000000000000000000 P 09f057fa17c842a1ab4f46fde4258923: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 01:01:41.164242 30885 master_runner.cc:186] Error getting master registration for 127.8.64.189:43859: OK, instance_id { permanent_uuid: "924b5ef69679445abd70279b32ec0554" instance_seqno: 1790211700439081 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 01:01:41.207487 30885 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 01:01:41.209404 31026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 924b5ef69679445abd70279b32ec0554: No bootstrap required, opened a new log
I20260924 01:01:41.220767 31026 raft_consensus.cc:348] T 00000000000000000000000000000000 P 924b5ef69679445abd70279b32ec0554 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90c9c5c753d4c8ab5e279460aaa7c06" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 43183 } } peers { permanent_uuid: "924b5ef69679445abd70279b32ec0554" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43859 } } peers { permanent_uuid: "09f057fa17c842a1ab4f46fde4258923" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36541 } }
I20260924 01:01:41.221280 31026 raft_consensus.cc:729] T 00000000000000000000000000000000 P 924b5ef69679445abd70279b32ec0554 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 924b5ef69679445abd70279b32ec0554, State: Initialized, Role: FOLLOWER
I20260924 01:01:41.221843 31026 consensus_queue.cc:260] T 00000000000000000000000000000000 P 924b5ef69679445abd70279b32ec0554 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90c9c5c753d4c8ab5e279460aaa7c06" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 43183 } } peers { permanent_uuid: "924b5ef69679445abd70279b32ec0554" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43859 } } peers { permanent_uuid: "09f057fa17c842a1ab4f46fde4258923" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36541 } }
I20260924 01:01:41.236295 31075 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 09f057fa17c842a1ab4f46fde4258923 [term 1 FOLLOWER]: Refusing update from remote peer b90c9c5c753d4c8ab5e279460aaa7c06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 01:01:41.237511 31134 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b90c9c5c753d4c8ab5e279460aaa7c06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09f057fa17c842a1ab4f46fde4258923" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 01:01:41.275863 31030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:41.191407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52982 (local address 127.8.64.189:43859)
0924 01:01:41.191633 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:41.191642 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:41.207107 (+ 15465us) server_negotiation.cc:407] Connection header received
0924 01:01:41.207306 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:41.207314 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:41.207396 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:41.207513 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:41.212048 (+ 4535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:41.213499 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:41.220471 (+ 6972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:41.221178 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:41.221288 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:41.254605 (+ 33317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:41.254613 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:41.254618 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:01:41.254674 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:41.259088 (+ 4414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:41.271437 (+ 12349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:41.271447 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:41.271453 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:41.271697 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:41.272690 (+ 993us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:41.272698 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:41.272704 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:41.273264 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:41.273624 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:41.274928 (+ 1304us) server_negotiation.cc:299] Negotiation successful
0924 01:01:41.275301 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260924 01:01:41.292711 31108 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09f057fa17c842a1ab4f46fde4258923 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b90c9c5c753d4c8ab5e279460aaa7c06. Latest consensus state: current_term: 1 leader_uuid: "b90c9c5c753d4c8ab5e279460aaa7c06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90c9c5c753d4c8ab5e279460aaa7c06" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 43183 } } peers { permanent_uuid: "924b5ef69679445abd70279b32ec0554" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43859 } } peers { permanent_uuid: "09f057fa17c842a1ab4f46fde4258923" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36541 } } }
I20260924 01:01:41.293063 31108 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09f057fa17c842a1ab4f46fde4258923 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 01:01:41.297719 31145 mvcc.cc:205] Tried to move back new op lower bound from 7332707128234602496 to 7332707127868157952. Current Snapshot: MvccSnapshot[applied={T|T < 7332707128234602496}]
I20260924 01:01:41.306069 31122 sys_catalog.cc:455] T 00000000000000000000000000000000 P b90c9c5c753d4c8ab5e279460aaa7c06 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b90c9c5c753d4c8ab5e279460aaa7c06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90c9c5c753d4c8ab5e279460aaa7c06" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 43183 } } peers { permanent_uuid: "924b5ef69679445abd70279b32ec0554" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43859 } } peers { permanent_uuid: "09f057fa17c842a1ab4f46fde4258923" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36541 } } }
I20260924 01:01:41.306432 31122 sys_catalog.cc:458] T 00000000000000000000000000000000 P b90c9c5c753d4c8ab5e279460aaa7c06 [sys.catalog]: This master's current role is: LEADER
I20260924 01:01:41.307426 31108 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09f057fa17c842a1ab4f46fde4258923 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b90c9c5c753d4c8ab5e279460aaa7c06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90c9c5c753d4c8ab5e279460aaa7c06" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 43183 } } peers { permanent_uuid: "924b5ef69679445abd70279b32ec0554" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43859 } } peers { permanent_uuid: "09f057fa17c842a1ab4f46fde4258923" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36541 } } }
I20260924 01:01:41.307685 31108 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09f057fa17c842a1ab4f46fde4258923 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 01:01:41.308176 31148 mvcc.cc:205] Tried to move back new op lower bound from 7332707128234602496 to 7332707127868157952. Current Snapshot: MvccSnapshot[applied={T|T < 7332707128234602496}]
I20260924 01:01:41.313830 31146 sys_catalog.cc:455] T 00000000000000000000000000000000 P 924b5ef69679445abd70279b32ec0554 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90c9c5c753d4c8ab5e279460aaa7c06" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 43183 } } peers { permanent_uuid: "924b5ef69679445abd70279b32ec0554" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43859 } } peers { permanent_uuid: "09f057fa17c842a1ab4f46fde4258923" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36541 } } }
I20260924 01:01:41.316812 31146 sys_catalog.cc:458] T 00000000000000000000000000000000 P 924b5ef69679445abd70279b32ec0554 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 01:01:41.316769 31001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b90c9c5c753d4c8ab5e279460aaa7c06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "924b5ef69679445abd70279b32ec0554"
I20260924 01:01:41.317286 31000 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 924b5ef69679445abd70279b32ec0554 [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:01:41.318011 31137 catalog_manager.cc:1347] Generated new cluster ID: 0b9e1725c0f042b5acb5d177c1d2e190
I20260924 01:01:41.318106 31137 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 01:01:41.320914 31000 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 924b5ef69679445abd70279b32ec0554 [term 1 FOLLOWER]: Refusing update from remote peer b90c9c5c753d4c8ab5e279460aaa7c06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 01:01:41.324080 31001 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 924b5ef69679445abd70279b32ec0554 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b90c9c5c753d4c8ab5e279460aaa7c06 in term 1.
I20260924 01:01:41.325639 31026 sys_catalog.cc:564] T 00000000000000000000000000000000 P 924b5ef69679445abd70279b32ec0554 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 01:01:41.326380 31134 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b90c9c5c753d4c8ab5e279460aaa7c06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "924b5ef69679445abd70279b32ec0554" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 01:01:41.344713 31137 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 01:01:41.346740 31137 catalog_manager.cc:1504] Loading token signing keys...
I20260924 01:01:41.355180 31106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:41.198168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53644 (local address 127.8.64.190:43183)
0924 01:01:41.198793 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:41.198799 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:41.222400 (+ 23601us) server_negotiation.cc:407] Connection header received
0924 01:01:41.223551 (+ 1151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:41.223558 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:41.223651 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:41.223727 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:41.249897 (+ 26170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:41.251087 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:41.279204 (+ 28117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:41.280236 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:41.280376 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:41.339505 (+ 59129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:41.339512 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:41.339517 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:01:41.339559 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:41.343357 (+ 3798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:41.350160 (+ 6803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:41.350167 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:41.350171 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:41.350353 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:41.351195 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:41.351201 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:41.351205 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:41.351567 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:41.351798 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:41.354536 (+ 2738us) server_negotiation.cc:299] Negotiation successful
0924 01:01:41.354813 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468}
I20260924 01:01:41.375487 31134 sys_catalog.cc:455] T 00000000000000000000000000000000 P b90c9c5c753d4c8ab5e279460aaa7c06 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b90c9c5c753d4c8ab5e279460aaa7c06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90c9c5c753d4c8ab5e279460aaa7c06" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 43183 } } peers { permanent_uuid: "924b5ef69679445abd70279b32ec0554" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43859 } } peers { permanent_uuid: "09f057fa17c842a1ab4f46fde4258923" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36541 } } }
I20260924 01:01:41.375762 31134 sys_catalog.cc:458] T 00000000000000000000000000000000 P b90c9c5c753d4c8ab5e279460aaa7c06 [sys.catalog]: This master's current role is: LEADER
I20260924 01:01:41.378360 31146 sys_catalog.cc:455] T 00000000000000000000000000000000 P 924b5ef69679445abd70279b32ec0554 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b90c9c5c753d4c8ab5e279460aaa7c06. Latest consensus state: current_term: 1 leader_uuid: "b90c9c5c753d4c8ab5e279460aaa7c06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90c9c5c753d4c8ab5e279460aaa7c06" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 43183 } } peers { permanent_uuid: "924b5ef69679445abd70279b32ec0554" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43859 } } peers { permanent_uuid: "09f057fa17c842a1ab4f46fde4258923" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36541 } } }
I20260924 01:01:41.378754 31146 sys_catalog.cc:458] T 00000000000000000000000000000000 P 924b5ef69679445abd70279b32ec0554 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 01:01:41.379138 31157 sys_catalog.cc:455] T 00000000000000000000000000000000 P 924b5ef69679445abd70279b32ec0554 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b90c9c5c753d4c8ab5e279460aaa7c06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90c9c5c753d4c8ab5e279460aaa7c06" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 43183 } } peers { permanent_uuid: "924b5ef69679445abd70279b32ec0554" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 43859 } } peers { permanent_uuid: "09f057fa17c842a1ab4f46fde4258923" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 36541 } } }
I20260924 01:01:41.379539 31157 sys_catalog.cc:458] T 00000000000000000000000000000000 P 924b5ef69679445abd70279b32ec0554 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 01:01:41.383071 31103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:41.224745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48578 (local address 127.8.64.188:36541)
0924 01:01:41.224938 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:41.224944 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:41.287286 (+ 62342us) server_negotiation.cc:407] Connection header received
0924 01:01:41.287508 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:41.287518 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:41.287628 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:41.287702 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:41.312054 (+ 24352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:41.313192 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:41.336208 (+ 23016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:41.336958 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:41.337075 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:41.359620 (+ 22545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:41.359627 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:41.359633 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:01:41.359678 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:41.363539 (+ 3861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:41.370767 (+ 7228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:41.370772 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:41.370776 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:41.370926 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:41.373067 (+ 2141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:41.373075 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:41.373083 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:41.373474 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:41.373758 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:41.382365 (+ 8607us) server_negotiation.cc:299] Negotiation successful
0924 01:01:41.382660 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260924 01:01:41.404935 31137 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b90c9c5c753d4c8ab5e279460aaa7c06: Generated new TSK 0
I20260924 01:01:41.406119 31137 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 01:01:41.493309 31167 catalog_manager.cc:1259] Loaded cluster ID: 0b9e1725c0f042b5acb5d177c1d2e190
I20260924 01:01:41.493434 31167 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 924b5ef69679445abd70279b32ec0554: loading cluster ID for follower catalog manager: success
I20260924 01:01:41.497318 31167 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 924b5ef69679445abd70279b32ec0554: acquiring CA information for follower catalog manager: success
I20260924 01:01:41.508198 31167 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 924b5ef69679445abd70279b32ec0554: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260924 01:01:41.535207 31106 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 01:01:41.515118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53650 (local address 127.8.64.190:43183)
0924 01:01:41.515442 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:41.515448 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:41.532985 (+ 17537us) server_negotiation.cc:407] Connection header received
0924 01:01:41.533513 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:41.533519 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:41.533613 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:41.533689 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:41.534911 (+ 1222us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53650: BlockingRecv error: recv got EOF from 127.0.0.1:53650 (error 108)
Metrics: {"server-negotiator.queue_time_us":169}
W20260924 01:01:41.537611 31030 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 01:01:41.517385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52998 (local address 127.8.64.189:43859)
0924 01:01:41.517578 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:41.517586 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:41.535988 (+ 18402us) server_negotiation.cc:407] Connection header received
0924 01:01:41.536248 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:41.536255 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:41.536378 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:41.536457 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:41.537376 (+ 919us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52998: BlockingRecv error: recv got EOF from 127.0.0.1:52998 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260924 01:01:41.541513 31103 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 01:01:41.539124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48588 (local address 127.8.64.188:36541)
0924 01:01:41.539318 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:41.539324 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:41.539878 (+ 554us) server_negotiation.cc:407] Connection header received
0924 01:01:41.540141 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:41.540147 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:41.540232 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:41.540301 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:41.541263 (+ 962us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48588: BlockingRecv error: recv got EOF from 127.0.0.1:48588 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260924 01:01:41.591719 31030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:41.574092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53006 (local address 127.8.64.189:43859)
0924 01:01:41.574288 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:41.574321 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:41.583442 (+ 9121us) server_negotiation.cc:407] Connection header received
0924 01:01:41.584023 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:41.584030 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:41.584264 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:41.584433 (+ 169us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 01:01:41.585208 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:41.586399 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:41.588473 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:41.589180 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:41.589283 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:41.590100 (+ 817us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 01:01:41.590857 (+ 757us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 01:01:41.590994 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:41.591255 (+ 261us) server_negotiation.cc:299] Negotiation successful
0924 01:01:41.591374 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260924 01:01:41.653028 31103 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 01:01:41.634283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48594 (local address 127.8.64.188:36541)
0924 01:01:41.634538 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:41.634545 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:41.650962 (+ 16417us) server_negotiation.cc:407] Connection header received
0924 01:01:41.651251 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:41.651264 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:41.651451 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:41.651549 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:41.652740 (+ 1191us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48594: BlockingRecv error: recv got EOF from 127.0.0.1:48594 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260924 01:01:41.713778 31030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:41.693849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53022 (local address 127.8.64.189:43859)
0924 01:01:41.694042 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:41.694049 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:41.706995 (+ 12946us) server_negotiation.cc:407] Connection header received
0924 01:01:41.707422 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:41.707430 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:41.707525 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:41.707595 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 01:01:41.708442 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:41.709610 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:41.711631 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:41.712249 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:41.712352 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:41.712933 (+ 581us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 01:01:41.713087 (+ 154us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 01:01:41.713143 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:41.713357 (+ 214us) server_negotiation.cc:299] Negotiation successful
0924 01:01:41.713443 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260924 01:01:41.760162 31103 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 01:01:41.750461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48606 (local address 127.8.64.188:36541)
0924 01:01:41.750648 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:41.750654 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:41.758115 (+ 7461us) server_negotiation.cc:407] Connection header received
0924 01:01:41.758672 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:41.758684 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:41.758781 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:41.758854 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:41.759891 (+ 1037us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48606: BlockingRecv error: recv got EOF from 127.0.0.1:48606 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260924 01:01:41.797173 31030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:41.791565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53026 (local address 127.8.64.189:43859)
0924 01:01:41.791820 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:41.791830 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:41.792351 (+ 521us) server_negotiation.cc:407] Connection header received
0924 01:01:41.792588 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:41.792593 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:41.792662 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:41.792761 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 01:01:41.793193 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:41.793958 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:41.795363 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:41.795816 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:41.795963 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:41.796373 (+ 410us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 01:01:41.796493 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 01:01:41.796591 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:41.796729 (+ 138us) server_negotiation.cc:299] Negotiation successful
0924 01:01:41.796817 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260924 01:01:41.808236 31103 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 01:01:41.803269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48610 (local address 127.8.64.188:36541)
0924 01:01:41.803537 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:41.803543 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:41.806779 (+ 3236us) server_negotiation.cc:407] Connection header received
0924 01:01:41.806997 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:41.807001 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:41.807073 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:41.807189 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:41.807967 (+ 778us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48610: BlockingRecv error: recv got EOF from 127.0.0.1:48610 (error 108)
Metrics: {"server-negotiator.queue_time_us":139}
I20260924 01:01:41.822050 31030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:41.815031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53042 (local address 127.8.64.189:43859)
0924 01:01:41.815331 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:41.815340 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:41.815902 (+ 562us) server_negotiation.cc:407] Connection header received
0924 01:01:41.816172 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:41.816177 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:41.816244 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:41.816346 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 01:01:41.816825 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:41.817825 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:41.820049 (+ 2224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:41.820739 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:41.820820 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:41.821333 (+ 513us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 01:01:41.821432 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 01:01:41.821533 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:41.821652 (+ 119us) server_negotiation.cc:299] Negotiation successful
0924 01:01:41.821730 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140}
W20260924 01:01:41.836249 31103 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 01:01:41.831840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48624 (local address 127.8.64.188:36541)
0924 01:01:41.832181 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:41.832190 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:41.834883 (+ 2693us) server_negotiation.cc:407] Connection header received
0924 01:01:41.835084 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:41.835090 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:41.835174 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:41.835279 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:41.836052 (+ 773us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48624: BlockingRecv error: recv got EOF from 127.0.0.1:48624 (error 108)
Metrics: {"server-negotiator.queue_time_us":180}
I20260924 01:01:41.853411 31030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:41.847866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53044 (local address 127.8.64.189:43859)
0924 01:01:41.848146 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:41.848156 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:41.848684 (+ 528us) server_negotiation.cc:407] Connection header received
0924 01:01:41.848895 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:41.848901 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:41.848980 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:41.849082 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 01:01:41.849520 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:41.850334 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:41.851642 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:41.852118 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:41.852248 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:41.852635 (+ 387us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 01:01:41.852749 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 01:01:41.852844 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:41.852979 (+ 135us) server_negotiation.cc:299] Negotiation successful
0924 01:01:41.853057 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260924 01:01:41.861666 31103 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 01:01:41.859627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48640 (local address 127.8.64.188:36541)
0924 01:01:41.859847 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:41.859856 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:41.860377 (+ 521us) server_negotiation.cc:407] Connection header received
0924 01:01:41.860569 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:41.860575 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:41.860655 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:41.860757 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:41.861469 (+ 712us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48640: BlockingRecv error: recv got EOF from 127.0.0.1:48640 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260924 01:01:41.887786 31030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:41.881755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53050 (local address 127.8.64.189:43859)
0924 01:01:41.882109 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:41.882119 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:41.882645 (+ 526us) server_negotiation.cc:407] Connection header received
0924 01:01:41.882985 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:41.882989 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:41.883063 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:41.883175 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 01:01:41.883578 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:41.884529 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:41.885821 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:41.886274 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:41.886452 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:41.887028 (+ 576us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 01:01:41.887137 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 01:01:41.887235 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:41.887373 (+ 138us) server_negotiation.cc:299] Negotiation successful
0924 01:01:41.887462 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260924 01:01:41.895666 31103 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 01:01:41.893568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48648 (local address 127.8.64.188:36541)
0924 01:01:41.893796 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:41.893805 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:41.894337 (+ 532us) server_negotiation.cc:407] Connection header received
0924 01:01:41.894547 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:41.894553 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:41.894640 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:41.894744 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:41.895479 (+ 735us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48648: BlockingRecv error: recv got EOF from 127.0.0.1:48648 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260924 01:01:41.937626 31030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:41.931660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53056 (local address 127.8.64.189:43859)
0924 01:01:41.931871 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:41.931877 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:41.932411 (+ 534us) server_negotiation.cc:407] Connection header received
0924 01:01:41.932735 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:41.932741 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:41.932859 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:41.932936 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 01:01:41.933620 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:41.934526 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:41.935840 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:41.936295 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:41.936433 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:41.936820 (+ 387us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 01:01:41.936916 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 01:01:41.937018 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:41.937165 (+ 147us) server_negotiation.cc:299] Negotiation successful
0924 01:01:41.937248 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260924 01:01:41.965866 31103 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 01:01:41.958707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48664 (local address 127.8.64.188:36541)
0924 01:01:41.958956 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:41.958965 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:41.964340 (+ 5375us) server_negotiation.cc:407] Connection header received
0924 01:01:41.964612 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:41.964618 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:41.964694 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:41.964801 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:41.965631 (+ 830us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48664: BlockingRecv error: recv got EOF from 127.0.0.1:48664 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
I20260924 01:01:42.039116 31030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:42.033120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53060 (local address 127.8.64.189:43859)
0924 01:01:42.033340 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:42.033348 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:42.033702 (+ 354us) server_negotiation.cc:407] Connection header received
0924 01:01:42.033892 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:42.033898 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:42.034016 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:42.034154 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 01:01:42.034585 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:42.035815 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:42.037132 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:42.037875 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:42.037995 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:42.038363 (+ 368us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 01:01:42.038497 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 01:01:42.038574 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:42.038714 (+ 140us) server_negotiation.cc:299] Negotiation successful
0924 01:01:42.038794 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260924 01:01:42.046448 31103 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 01:01:42.044595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48676 (local address 127.8.64.188:36541)
0924 01:01:42.044786 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:42.044792 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:42.045112 (+ 320us) server_negotiation.cc:407] Connection header received
0924 01:01:42.045253 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:42.045257 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:42.045358 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:42.045468 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:42.046206 (+ 738us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48676: BlockingRecv error: recv got EOF from 127.0.0.1:48676 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260924 01:01:42.075615 31119 catalog_manager.cc:1259] Loaded cluster ID: 0b9e1725c0f042b5acb5d177c1d2e190
I20260924 01:01:42.075713 31119 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 09f057fa17c842a1ab4f46fde4258923: loading cluster ID for follower catalog manager: success
I20260924 01:01:42.078150 31119 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 09f057fa17c842a1ab4f46fde4258923: acquiring CA information for follower catalog manager: success
I20260924 01:01:42.079924 31119 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 09f057fa17c842a1ab4f46fde4258923: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 01:01:42.184545 31030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:42.179480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53064 (local address 127.8.64.189:43859)
0924 01:01:42.179679 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:42.179686 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:42.180090 (+ 404us) server_negotiation.cc:407] Connection header received
0924 01:01:42.180242 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:42.180246 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:42.180356 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:42.180461 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 01:01:42.180803 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:42.181587 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:42.183045 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:42.183537 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:42.183639 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:42.183981 (+ 342us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 01:01:42.184091 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 01:01:42.184159 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:42.184275 (+ 116us) server_negotiation.cc:299] Negotiation successful
0924 01:01:42.184336 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260924 01:01:42.195165 31103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:42.189937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48684 (local address 127.8.64.188:36541)
0924 01:01:42.190176 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:42.190182 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:42.190454 (+ 272us) server_negotiation.cc:407] Connection header received
0924 01:01:42.190580 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:42.190583 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:42.190726 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:42.190838 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 01:01:42.191178 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:42.191843 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:42.193070 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:42.193461 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:42.193555 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:42.193974 (+ 419us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 01:01:42.194548 (+ 574us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 01:01:42.194664 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:42.194786 (+ 122us) server_negotiation.cc:299] Negotiation successful
0924 01:01:42.194853 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260924 01:01:42.208961 31272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:42.201459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53658 (local address 127.8.64.190:43183)
0924 01:01:42.202051 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:42.202057 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:42.202494 (+ 437us) server_negotiation.cc:407] Connection header received
0924 01:01:42.202745 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:42.202750 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:42.202891 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:42.203077 (+ 186us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 01:01:42.203784 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:42.204500 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:42.206585 (+ 2085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:42.207039 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:42.207168 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:42.207900 (+ 732us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 01:01:42.208299 (+ 399us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 01:01:42.208423 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:42.208556 (+ 133us) server_negotiation.cc:299] Negotiation successful
0924 01:01:42.208640 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":281,"threads_started":1}
I20260924 01:01:42.210745 31030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:42.200966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53076 (local address 127.8.64.189:43859)
0924 01:01:42.201275 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:42.201285 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:42.203240 (+ 1955us) server_negotiation.cc:407] Connection header received
0924 01:01:42.203566 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:42.203574 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:42.203689 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:42.203765 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 01:01:42.204758 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:42.205695 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:42.208101 (+ 2406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:42.208802 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:42.208939 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:42.209997 (+ 1058us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 01:01:42.210121 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 01:01:42.210213 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:42.210383 (+ 170us) server_negotiation.cc:299] Negotiation successful
0924 01:01:42.210481 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260924 01:01:42.211439 31103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:42.203391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48696 (local address 127.8.64.188:36541)
0924 01:01:42.203612 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:42.203619 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:42.205188 (+ 1569us) server_negotiation.cc:407] Connection header received
0924 01:01:42.205358 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:42.205364 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:42.205440 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:42.205529 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 01:01:42.206221 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:42.207304 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:42.209400 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:42.209905 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:42.210028 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:42.210587 (+ 559us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 01:01:42.210700 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 01:01:42.210769 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:42.211048 (+ 279us) server_negotiation.cc:299] Negotiation successful
0924 01:01:42.211137 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260924 01:01:42.214435 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 30885
I20260924 01:01:42.215489 30947 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 01:01:42.323371 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 30885
W20260924 01:01:42.344560 30975 connection.cc:535] server connection from 127.0.0.1:52964 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260924 01:01:42.345151 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 30959
I20260924 01:01:42.346125 31021 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 01:01:42.451908 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 30959
I20260924 01:01:42.473907 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 31033
I20260924 01:01:42.474975 31095 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 01:01:42.581102 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 31033
2026-09-24T01:01:42Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4618 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 01:01:42 dist-test-slave-mmhs krb5kdc[31279](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 01:01:42 dist-test-slave-mmhs krb5kdc[31279](info): set up 2 sockets
Sep 24 01:01:42 dist-test-slave-mmhs krb5kdc[31279](info): commencing operation
krb5kdc: starting...
W20260924 01:01:43.704706 8450 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.000s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/krb5kdc/test-user.keytab.
Sep 24 01:01:43 dist-test-slave-mmhs krb5kdc[31279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211703, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T01:01:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T01:01:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.64.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
Entry for principal kudu/127.8.64.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab.
I20260924 01:01:43.981828 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:45487
--webserver_interface=127.8.64.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:40845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:45487,127.8.64.189:45161,127.8.64.188:33393
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 01:01:44.309438 31292 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 01:01:44.309798 31292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:01:44.309934 31292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:01:44.310045 31292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:01:44.317359 31292 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 01:01:44.317452 31292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:01:44.317554 31292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:01:44.317628 31292 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 01:01:44.317719 31292 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 01:01:44.327535 31292 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:40845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:45487,127.8.64.189:45161,127.8.64.188:33393
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.190.keytab
--principal=kudu/127.8.64.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.190:45487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.64.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:01:44.328825 31292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:01:44.330148 31292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:01:44.342180 31298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:44.342221 31297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:44.342264 31300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:01:44.342983 31292 server_base.cc:1034] running on GCE node
I20260924 01:01:44.343724 31292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:01:44.345084 31292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:01:44.346365 31292 hybrid_clock.cc:648] HybridClock initialized: now 1790211704346253 us; error 43 us; skew 500 ppm
I20260924 01:01:44.346688 31292 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 01:01:44 dist-test-slave-mmhs krb5kdc[31279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211704, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 01:01:44.349689 31292 init.cc:377] Logged in from keytab as kudu/127.8.64.190@KRBTEST.COM (short username kudu)
I20260924 01:01:44.351197 31292 webserver.cc:460] Webserver started at http://127.8.64.190:41087/ using document root <none> and password file <none>
I20260924 01:01:44.351859 31292 fs_manager.cc:359] Metadata directory not provided
I20260924 01:01:44.351986 31292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:01:44.352301 31292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:01:44.355276 31292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-0/data/instance:
uuid: "a7eb9683ee6345649db8d50f4288fe71"
format_stamp: "Formatted at 2026-09-24 01:01:44 on dist-test-slave-mmhs"
server_key: "ffb6f2fbf6c09107993095bd64f41a99"
server_key_iv: "46d591506ff01884d76bb980d554ac1d"
server_key_version: "encryptionkey@0"
I20260924 01:01:44.356050 31292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-0/wal/instance:
uuid: "a7eb9683ee6345649db8d50f4288fe71"
format_stamp: "Formatted at 2026-09-24 01:01:44 on dist-test-slave-mmhs"
server_key: "ffb6f2fbf6c09107993095bd64f41a99"
server_key_iv: "46d591506ff01884d76bb980d554ac1d"
server_key_version: "encryptionkey@0"
I20260924 01:01:44.361048 31292 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260924 01:01:44.364449 31307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:01:44.365756 31292 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260924 01:01:44.365934 31292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-0/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-0/wal
uuid: "a7eb9683ee6345649db8d50f4288fe71"
format_stamp: "Formatted at 2026-09-24 01:01:44 on dist-test-slave-mmhs"
server_key: "ffb6f2fbf6c09107993095bd64f41a99"
server_key_iv: "46d591506ff01884d76bb980d554ac1d"
server_key_version: "encryptionkey@0"
I20260924 01:01:44.366149 31292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:01:44.381757 31292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:01:44.386085 31292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:01:44.386415 31292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:01:44.405294 31292 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.190:45487
I20260924 01:01:44.405321 31358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.190:45487 every 8 connection(s)
I20260924 01:01:44.406677 31292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-0/data/info.pb
I20260924 01:01:44.407497 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 31292
I20260924 01:01:44.407701 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-0/wal/instance
I20260924 01:01:44.408300 8450 external_mini_cluster.cc:1428] Setting key d59cd8d1dceabb2db31abf974ede30b3
I20260924 01:01:44.411780 31359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:01:44.421572 31359 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45487 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.64.189@KRBTEST.COM; defaulting to no policy
Sep 24 01:01:44 dist-test-slave-mmhs krb5kdc[31279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211704, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.189@KRBTEST.COM" created.
I20260924 01:01:44.445720 31363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:44.424673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45487 (local address 127.0.0.1:53798)
0924 01:01:44.425963 (+ 1290us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:44.426026 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0924 01:01:44.427782 (+ 1756us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:01:44.428724 (+ 942us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:01:44.428728 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:01:44.429298 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0924 01:01:44.429641 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:44.429658 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:44.432046 (+ 2388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:44.432050 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:44.433269 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:44.433281 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:44.434710 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:44.434715 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:44.434864 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:44.435626 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 01:01:44.435649 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0924 01:01:44.437880 (+ 2231us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 01:01:44.441818 (+ 3938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:01:44.441827 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:01:44.441836 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:01:44.442241 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:01:44.442763 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:01:44.442767 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:01:44.442769 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:01:44.442935 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:01:44.443868 (+ 933us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 01:01:44.443873 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 01:01:44.444041 (+ 168us) client_negotiation.cc:769] Sending connection context
0924 01:01:44.444251 (+ 210us) client_negotiation.cc:240] Negotiation successful
0924 01:01:44.444632 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1034,"mutex_wait_us":40,"thread_start_us":815,"threads_started":1}
I20260924 01:01:44.445748 31362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:44.424673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53798 (local address 127.8.64.190:45487)
0924 01:01:44.425578 (+ 905us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:44.425594 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:44.426160 (+ 566us) server_negotiation.cc:407] Connection header received
0924 01:01:44.428139 (+ 1979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:44.428149 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:44.428465 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:44.428694 (+ 229us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:44.429850 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:44.431872 (+ 2022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:44.433414 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:44.434489 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:44.434722 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:44.438169 (+ 3447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:44.438178 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:44.438191 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:01:44.438220 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:44.441604 (+ 3384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:44.442442 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:44.442454 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:44.442460 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:44.442647 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:44.443044 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:44.443051 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:44.443057 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:44.443733 (+ 676us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:44.443936 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:44.444414 (+ 478us) server_negotiation.cc:299] Negotiation successful
0924 01:01:44.444800 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":142,"server-negotiator.queue_time_us":630,"thread_start_us":388,"threads_started":1}
I20260924 01:01:44.448616 31359 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.64.189" port: 45161 } has no permanent_uuid. Determining permanent_uuid...
I20260924 01:01:44.449714 31363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:44.449234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:45161 (local address 127.0.0.1:46122)
0924 01:01:44.449443 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:44.449569 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.189:45161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260924 01:01:44.449959 31310 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.64.189:45161: connect: Connection refused (error 111)
I20260924 01:01:44.451692 31363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:44.451248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:45161 (local address 127.0.0.1:46132)
0924 01:01:44.451414 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:44.451520 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.189:45161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260924 01:01:44.451920 31359 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.64.189:45161: Network error: Client connection negotiation failed: client connection to 127.8.64.189:45161: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.189.keytab.
Entry for principal kudu/127.8.64.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.189.keytab.
I20260924 01:01:44.474118 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.189:45161
--webserver_interface=127.8.64.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.189.keytab
--principal=kudu/127.8.64.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:40845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:45487,127.8.64.189:45161,127.8.64.188:33393
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 01:01:44.501299 31359 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.64.189" port: 45161 } attempt: 1
I20260924 01:01:44.502516 31363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:44.501988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:45161 (local address 127.0.0.1:46142)
0924 01:01:44.502202 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:44.502355 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.189:45161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260924 01:01:44.503517 31363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:44.503140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:45161 (local address 127.0.0.1:46154)
0924 01:01:44.503283 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:44.503364 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.189:45161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260924 01:01:44.503736 31359 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.64.189:45161: Network error: Client connection negotiation failed: client connection to 127.8.64.189:45161: connect: Connection refused (error 111)
I20260924 01:01:44.572006 31359 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.64.189" port: 45161 } attempt: 2
I20260924 01:01:44.573401 31363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:44.572755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:45161 (local address 127.0.0.1:46164)
0924 01:01:44.573044 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:44.573174 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.189:45161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260924 01:01:44.574821 31363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:44.574238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:45161 (local address 127.0.0.1:46178)
0924 01:01:44.574489 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:44.574601 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.189:45161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
W20260924 01:01:44.575037 31359 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.64.189:45161: Network error: Client connection negotiation failed: client connection to 127.8.64.189:45161: connect: Connection refused (error 111)
I20260924 01:01:44.666404 31359 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.64.189" port: 45161 } attempt: 3
I20260924 01:01:44.667704 31363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:44.667090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:45161 (local address 127.0.0.1:46182)
0924 01:01:44.667356 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:44.667500 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.189:45161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260924 01:01:44.668982 31363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:44.668465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:45161 (local address 127.0.0.1:46188)
0924 01:01:44.668652 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:44.668758 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.189:45161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260924 01:01:44.669216 31359 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.64.189:45161: Network error: Client connection negotiation failed: client connection to 127.8.64.189:45161: connect: Connection refused (error 111)
W20260924 01:01:44.808035 31366 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 01:01:44.808475 31366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:01:44.808586 31366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:01:44.808727 31366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260924 01:01:44.812480 31359 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.64.189" port: 45161 } attempt: 4
I20260924 01:01:44.813766 31363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:44.813176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:45161 (local address 127.0.0.1:46192)
0924 01:01:44.813490 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:44.813600 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.189:45161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151}
I20260924 01:01:44.814875 31363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:44.814489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:45161 (local address 127.0.0.1:46200)
0924 01:01:44.814638 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:44.814730 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.189:45161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260924 01:01:44.815056 31359 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.64.189:45161: Network error: Client connection negotiation failed: client connection to 127.8.64.189:45161: connect: Connection refused (error 111)
W20260924 01:01:44.817281 31366 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 01:01:44.817364 31366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:01:44.817478 31366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:01:44.817553 31366 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 01:01:44.817641 31366 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 01:01:44.828239 31366 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:40845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:45487,127.8.64.189:45161,127.8.64.188:33393
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.189.keytab
--principal=kudu/127.8.64.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.189:45161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.64.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:01:44.829560 31366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:01:44.830976 31366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:01:44.842882 31372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:44.842926 31374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:44.843000 31371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:01:44.843577 31366 server_base.cc:1034] running on GCE node
I20260924 01:01:44.844350 31366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:01:44.845628 31366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:01:44.846851 31366 hybrid_clock.cc:648] HybridClock initialized: now 1790211704846769 us; error 38 us; skew 500 ppm
I20260924 01:01:44.847177 31366 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 01:01:44 dist-test-slave-mmhs krb5kdc[31279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211704, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 01:01:44.850231 31366 init.cc:377] Logged in from keytab as kudu/127.8.64.189@KRBTEST.COM (short username kudu)
I20260924 01:01:44.851626 31366 webserver.cc:460] Webserver started at http://127.8.64.189:32857/ using document root <none> and password file <none>
I20260924 01:01:44.852314 31366 fs_manager.cc:359] Metadata directory not provided
I20260924 01:01:44.852445 31366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:01:44.852749 31366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:01:44.855731 31366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-1/data/instance:
uuid: "c3f2125d04bf42eeb6c7099439dcdf1c"
format_stamp: "Formatted at 2026-09-24 01:01:44 on dist-test-slave-mmhs"
server_key: "5a0e1b6d29142a512222b59c61101783"
server_key_iv: "b8a190b4c8ede357342521f376b46321"
server_key_version: "encryptionkey@0"
I20260924 01:01:44.856524 31366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-1/wal/instance:
uuid: "c3f2125d04bf42eeb6c7099439dcdf1c"
format_stamp: "Formatted at 2026-09-24 01:01:44 on dist-test-slave-mmhs"
server_key: "5a0e1b6d29142a512222b59c61101783"
server_key_iv: "b8a190b4c8ede357342521f376b46321"
server_key_version: "encryptionkey@0"
I20260924 01:01:44.861538 31366 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260924 01:01:44.864993 31381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:01:44.866492 31366 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260924 01:01:44.866689 31366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-1/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-1/wal
uuid: "c3f2125d04bf42eeb6c7099439dcdf1c"
format_stamp: "Formatted at 2026-09-24 01:01:44 on dist-test-slave-mmhs"
server_key: "5a0e1b6d29142a512222b59c61101783"
server_key_iv: "b8a190b4c8ede357342521f376b46321"
server_key_version: "encryptionkey@0"
I20260924 01:01:44.866901 31366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:01:44.892007 31366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:01:44.896437 31366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:01:44.896734 31366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:01:44.916018 31366 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.189:45161
I20260924 01:01:44.916054 31432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.189:45161 every 8 connection(s)
I20260924 01:01:44.917398 31366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-1/data/info.pb
I20260924 01:01:44.921137 31433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:01:44.921506 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 31366
I20260924 01:01:44.921707 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-1/wal/instance
I20260924 01:01:44.922123 8450 external_mini_cluster.cc:1428] Setting key 70243147033e007b08089fb64b3a3da9
I20260924 01:01:44.927731 31433 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45487 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 01:01:44 dist-test-slave-mmhs krb5kdc[31279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211704, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.189@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.64.188@KRBTEST.COM; defaulting to no policy
I20260924 01:01:44.950968 31362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:44.930334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53806 (local address 127.8.64.190:45487)
0924 01:01:44.930550 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:44.930556 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:44.931836 (+ 1280us) server_negotiation.cc:407] Connection header received
0924 01:01:44.934865 (+ 3029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:44.934869 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:44.934942 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:44.935035 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:44.937209 (+ 2174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:44.937892 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:44.940432 (+ 2540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:44.940869 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:44.940973 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:44.945187 (+ 4214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:44.945192 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:44.945196 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:01:44.945229 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:44.947923 (+ 2694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:44.948653 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:44.948658 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:44.948660 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:44.948775 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:44.949180 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:44.949185 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:44.949188 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:44.949472 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:44.949645 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:44.950406 (+ 761us) server_negotiation.cc:299] Negotiation successful
0924 01:01:44.950639 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260924 01:01:44.952059 31436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:44.930811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45487 (local address 127.0.0.1:53806)
0924 01:01:44.931648 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:44.931709 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0924 01:01:44.934556 (+ 2847us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:01:44.935288 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:01:44.935312 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:01:44.936123 (+ 811us) client_negotiation.cc:189] Negotiated authn=SASL
0924 01:01:44.936981 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:44.937009 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:44.938059 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:44.938067 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:44.940224 (+ 2157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:44.940241 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:44.941097 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:44.941107 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:44.941454 (+ 347us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:44.942203 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 01:01:44.942237 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0924 01:01:44.944948 (+ 2711us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 01:01:44.948064 (+ 3116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:01:44.948073 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:01:44.948090 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:01:44.948525 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:01:44.948882 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:01:44.948887 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:01:44.948891 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:01:44.949067 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:01:44.949632 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 01:01:44.949640 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 01:01:44.949998 (+ 358us) client_negotiation.cc:769] Sending connection context
0924 01:01:44.950267 (+ 269us) client_negotiation.cc:240] Negotiation successful
0924 01:01:44.950833 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":318,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.64.188@KRBTEST.COM" created.
I20260924 01:01:44.953872 31433 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.64.189" port: 45161 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 01:01:44 dist-test-slave-mmhs krb5kdc[31279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211704, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.189@KRBTEST.COM for kudu/127.8.64.189@KRBTEST.COM
I20260924 01:01:44.967689 31436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:44.954509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:45161 (local address 127.0.0.1:46208)
0924 01:01:44.954702 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:44.954720 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0924 01:01:44.954802 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:01:44.955847 (+ 1045us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:01:44.955851 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:01:44.956291 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0924 01:01:44.956418 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:44.956426 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:44.957769 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:44.957777 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:44.958720 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:44.958732 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:44.959497 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:44.959502 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:44.959682 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:44.960226 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 01:01:44.960242 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0924 01:01:44.962153 (+ 1911us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 01:01:44.965200 (+ 3047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:01:44.965207 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:01:44.965211 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:01:44.965554 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:01:44.965913 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:01:44.965918 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:01:44.965922 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:01:44.966054 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:01:44.966711 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 01:01:44.966716 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 01:01:44.966841 (+ 125us) client_negotiation.cc:769] Sending connection context
0924 01:01:44.967005 (+ 164us) client_negotiation.cc:240] Negotiation successful
0924 01:01:44.967279 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260924 01:01:44.967792 31437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:44.954830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46208 (local address 127.8.64.189:45161)
0924 01:01:44.955464 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:44.955472 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:44.955496 (+ 24us) server_negotiation.cc:407] Connection header received
0924 01:01:44.955537 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:44.955542 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:44.955710 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:44.955862 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:44.956588 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:44.957644 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:44.958843 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:44.959357 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:44.959483 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:44.962368 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:44.962379 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:44.962392 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:01:44.962419 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:44.965064 (+ 2645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:44.965674 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:44.965682 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:44.965686 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:44.965795 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:44.966148 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:44.966155 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:44.966159 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:44.966574 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:44.966775 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:44.967134 (+ 359us) server_negotiation.cc:299] Negotiation successful
0924 01:01:44.967389 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":491,"thread_start_us":364,"threads_started":1}
I20260924 01:01:44.970232 31433 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.64.188" port: 33393 } has no permanent_uuid. Determining permanent_uuid...
I20260924 01:01:44.971581 31436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:44.971102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:33393 (local address 127.0.0.1:55272)
0924 01:01:44.971301 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:44.971429 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:33393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260924 01:01:44.971843 31383 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.64.188:33393: connect: Connection refused (error 111)
I20260924 01:01:44.973351 31436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:44.973040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:33393 (local address 127.0.0.1:55280)
0924 01:01:44.973184 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:44.973255 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:33393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260924 01:01:44.973599 31433 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.64.188:33393: Network error: Client connection negotiation failed: client connection to 127.8.64.188:33393: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.64.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.188.keytab.
Entry for principal kudu/127.8.64.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.188.keytab.
I20260924 01:01:44.985656 8450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
/tmp/dist-test-taskDpZYBj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.188:33393
--webserver_interface=127.8.64.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.188.keytab
--principal=kudu/127.8.64.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.64.148:40845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.64.190:45487,127.8.64.189:45161,127.8.64.188:33393
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 01:01:45.022873 31433 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.64.188" port: 33393 } attempt: 1
I20260924 01:01:45.023957 31436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:45.023524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:33393 (local address 127.0.0.1:55284)
0924 01:01:45.023711 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:45.023814 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:33393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260924 01:01:45.024902 31436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:45.024592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:33393 (local address 127.0.0.1:55296)
0924 01:01:45.024728 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:45.024812 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:33393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260924 01:01:45.025151 31433 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.64.188:33393: Network error: Client connection negotiation failed: client connection to 127.8.64.188:33393: connect: Connection refused (error 111)
I20260924 01:01:45.093379 31433 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.64.188" port: 33393 } attempt: 2
I20260924 01:01:45.094549 31436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:45.094048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:33393 (local address 127.0.0.1:55312)
0924 01:01:45.094247 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:45.094393 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:33393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260924 01:01:45.095536 31436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:45.095210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:33393 (local address 127.0.0.1:55328)
0924 01:01:45.095353 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:45.095421 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:33393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260924 01:01:45.095741 31433 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.64.188:33393: Network error: Client connection negotiation failed: client connection to 127.8.64.188:33393: connect: Connection refused (error 111)
I20260924 01:01:45.114334 31359 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.64.189" port: 45161 } attempt: 5
Sep 24 01:01:45 dist-test-slave-mmhs krb5kdc[31279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211704, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for kudu/127.8.64.189@KRBTEST.COM
I20260924 01:01:45.127256 31437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:45.115157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46222 (local address 127.8.64.189:45161)
0924 01:01:45.115375 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:45.115383 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:45.115412 (+ 29us) server_negotiation.cc:407] Connection header received
0924 01:01:45.115642 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:45.115647 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:45.115710 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:45.115817 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:45.116659 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:45.117348 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:45.118591 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:45.119032 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:45.119173 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:45.122160 (+ 2987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:45.122165 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:45.122168 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:01:45.122198 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:45.124801 (+ 2603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:45.125411 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:45.125418 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:45.125422 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:45.125542 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:45.125907 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:45.125912 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:45.125915 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:45.126183 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:45.126361 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:45.126736 (+ 375us) server_negotiation.cc:299] Negotiation successful
0924 01:01:45.126941 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260924 01:01:45.127262 31363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:45.115125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:45161 (local address 127.0.0.1:46222)
0924 01:01:45.115337 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:45.115358 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0924 01:01:45.115492 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:01:45.115865 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:01:45.115872 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:01:45.116313 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0924 01:01:45.116514 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:45.116524 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:45.117477 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:45.117483 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:45.118451 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:45.118464 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:45.119188 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:45.119194 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:45.119392 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:45.119993 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 01:01:45.120012 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0924 01:01:45.121977 (+ 1965us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 01:01:45.124934 (+ 2957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:01:45.124940 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:01:45.124944 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:01:45.125297 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:01:45.125649 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:01:45.125654 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:01:45.125658 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:01:45.125789 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:01:45.126323 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 01:01:45.126328 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 01:01:45.126489 (+ 161us) client_negotiation.cc:769] Sending connection context
0924 01:01:45.126630 (+ 141us) client_negotiation.cc:240] Negotiation successful
0924 01:01:45.126916 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260924 01:01:45.128651 31359 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.64.188" port: 33393 } has no permanent_uuid. Determining permanent_uuid...
I20260924 01:01:45.129685 31363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:45.129316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:33393 (local address 127.0.0.1:55334)
0924 01:01:45.129474 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:45.129555 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:33393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260924 01:01:45.130600 31363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:45.130253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:33393 (local address 127.0.0.1:55336)
0924 01:01:45.130438 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:45.130510 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:33393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
W20260924 01:01:45.130818 31359 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.64.188:33393: Network error: Client connection negotiation failed: client connection to 127.8.64.188:33393: connect: Connection refused (error 111)
I20260924 01:01:45.182066 31359 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.64.188" port: 33393 } attempt: 1
I20260924 01:01:45.183346 31363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:45.182771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:33393 (local address 127.0.0.1:55340)
0924 01:01:45.183012 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:45.183132 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:33393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260924 01:01:45.184396 31363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:45.184006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:33393 (local address 127.0.0.1:55346)
0924 01:01:45.184161 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:45.184249 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:33393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260924 01:01:45.184618 31359 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.64.188:33393: Network error: Client connection negotiation failed: client connection to 127.8.64.188:33393: connect: Connection refused (error 111)
I20260924 01:01:45.186923 31433 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.64.188" port: 33393 } attempt: 3
I20260924 01:01:45.187844 31436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:45.187431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:33393 (local address 127.0.0.1:55358)
0924 01:01:45.187633 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:45.187711 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:33393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260924 01:01:45.188860 31436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:45.188471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:33393 (local address 127.0.0.1:55364)
0924 01:01:45.188697 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:45.188763 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:33393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
W20260924 01:01:45.189110 31433 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.64.188:33393: Network error: Client connection negotiation failed: client connection to 127.8.64.188:33393: connect: Connection refused (error 111)
I20260924 01:01:45.252856 31359 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.64.188" port: 33393 } attempt: 2
I20260924 01:01:45.254130 31363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:45.253546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:33393 (local address 127.0.0.1:55370)
0924 01:01:45.253812 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:45.253936 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:33393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260924 01:01:45.255175 31363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:45.254780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:33393 (local address 127.0.0.1:55378)
0924 01:01:45.254940 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:45.255034 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:33393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260924 01:01:45.255388 31359 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.64.188:33393: Network error: Client connection negotiation failed: client connection to 127.8.64.188:33393: connect: Connection refused (error 111)
W20260924 01:01:45.316612 31440 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 01:01:45.316928 31440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 01:01:45.317000 31440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 01:01:45.317076 31440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 01:01:45.324339 31440 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 01:01:45.324438 31440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 01:01:45.324494 31440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 01:01:45.324545 31440 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 01:01:45.324599 31440 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 01:01:45.332414 31433 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.64.188" port: 33393 } attempt: 4
I20260924 01:01:45.333515 31436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:45.333058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:33393 (local address 127.0.0.1:55382)
0924 01:01:45.333280 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:45.333363 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:33393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260924 01:01:45.334403 31440 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.64.148:40845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.64.190:45487,127.8.64.189:45161,127.8.64.188:33393
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/krb5kdc/kudu_127.8.64.188.keytab
--principal=kudu/127.8.64.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.64.188:33393
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.64.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 23:51:06 UTC on ccycloud.quasar-ikldij.root.comops.site
build id 4669120
ASAN enabled
I20260924 01:01:45.334483 31436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:45.334138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:33393 (local address 127.0.0.1:55394)
0924 01:01:45.334272 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:45.334386 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:33393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260924 01:01:45.334792 31433 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.64.188:33393: Network error: Client connection negotiation failed: client connection to 127.8.64.188:33393: connect: Connection refused (error 111)
I20260924 01:01:45.335728 31440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 01:01:45.337028 31440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 01:01:45.348404 31446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:45.348378 31448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 01:01:45.348459 31445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 01:01:45.349459 31440 server_base.cc:1034] running on GCE node
I20260924 01:01:45.350354 31440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 01:01:45.351848 31440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 01:01:45.353088 31440 hybrid_clock.cc:648] HybridClock initialized: now 1790211705353009 us; error 56 us; skew 500 ppm
I20260924 01:01:45.353471 31440 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 24 01:01:45 dist-test-slave-mmhs krb5kdc[31279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211705, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 01:01:45.356709 31440 init.cc:377] Logged in from keytab as kudu/127.8.64.188@KRBTEST.COM (short username kudu)
I20260924 01:01:45.358174 31440 webserver.cc:460] Webserver started at http://127.8.64.188:35973/ using document root <none> and password file <none>
I20260924 01:01:45.358903 31440 fs_manager.cc:359] Metadata directory not provided
I20260924 01:01:45.359031 31440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 01:01:45.359344 31440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 01:01:45.361609 31359 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.64.188" port: 33393 } attempt: 3
I20260924 01:01:45.362785 31363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:45.362193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:33393 (local address 127.0.0.1:55406)
0924 01:01:45.362474 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:45.362579 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:33393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260924 01:01:45.362489 31440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-2/data/instance:
uuid: "db82d452228b45cca1b2b4ce1a8a2ae0"
format_stamp: "Formatted at 2026-09-24 01:01:45 on dist-test-slave-mmhs"
server_key: "41560efe4b35f0d6ffc84764d4760ef7"
server_key_iv: "b4734d5d1870e8b080893be1c0e7f20a"
server_key_version: "encryptionkey@0"
I20260924 01:01:45.363451 31440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-2/wal/instance:
uuid: "db82d452228b45cca1b2b4ce1a8a2ae0"
format_stamp: "Formatted at 2026-09-24 01:01:45 on dist-test-slave-mmhs"
server_key: "41560efe4b35f0d6ffc84764d4760ef7"
server_key_iv: "b4734d5d1870e8b080893be1c0e7f20a"
server_key_version: "encryptionkey@0"
I20260924 01:01:45.363878 31363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:45.363414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:33393 (local address 127.0.0.1:55410)
0924 01:01:45.363621 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:45.363722 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.64.188:33393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
W20260924 01:01:45.364138 31359 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.64.188:33393: Network error: Client connection negotiation failed: client connection to 127.8.64.188:33393: connect: Connection refused (error 111)
I20260924 01:01:45.368983 31440 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260924 01:01:45.372362 31455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 01:01:45.373685 31440 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260924 01:01:45.373869 31440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-2/data,/tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-2/wal
uuid: "db82d452228b45cca1b2b4ce1a8a2ae0"
format_stamp: "Formatted at 2026-09-24 01:01:45 on dist-test-slave-mmhs"
server_key: "41560efe4b35f0d6ffc84764d4760ef7"
server_key_iv: "b4734d5d1870e8b080893be1c0e7f20a"
server_key_version: "encryptionkey@0"
I20260924 01:01:45.374076 31440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 01:01:45.404188 31440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 01:01:45.408344 31440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 01:01:45.408622 31440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 01:01:45.426677 31440 rpc_server.cc:307] RPC server started. Bound to: 127.8.64.188:33393
I20260924 01:01:45.426697 31506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.64.188:33393 every 8 connection(s)
I20260924 01:01:45.427937 31440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-2/data/info.pb
I20260924 01:01:45.431552 31507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 01:01:45.431910 8450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu as pid 31440
I20260924 01:01:45.432096 8450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDpZYBj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790211343099518-8450-0/minicluster-data/master-2/wal/instance
I20260924 01:01:45.432655 8450 external_mini_cluster.cc:1428] Setting key 6b7c24d4611fdafcd5e26d4efe5c24dd
I20260924 01:01:45.440142 31507 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45487 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 01:01:45 dist-test-slave-mmhs krb5kdc[31279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211703, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.188@KRBTEST.COM
Sep 24 01:01:45 dist-test-slave-mmhs krb5kdc[31279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211705, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.188@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 01:01:45.456692 31510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:45.434419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55426 (local address 127.8.64.188:33393)
0924 01:01:45.435233 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:45.435245 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:45.435284 (+ 39us) server_negotiation.cc:407] Connection header received
0924 01:01:45.437578 (+ 2294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:45.437600 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:45.438011 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:45.438474 (+ 463us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:45.439910 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:45.441231 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:45.442610 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:45.443664 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:45.443843 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:45.448566 (+ 4723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:45.448574 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:45.448585 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:01:45.448657 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:45.452177 (+ 3520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:45.452924 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:45.452935 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:45.452945 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:45.453122 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:45.453524 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:45.453530 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:45.453536 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:45.454235 (+ 699us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:45.454596 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:45.454969 (+ 373us) server_negotiation.cc:299] Negotiation successful
0924 01:01:45.455387 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620,"thread_start_us":394,"threads_started":1}
I20260924 01:01:45.458526 31511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:45.442449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.190:45487 (local address 127.0.0.1:53812)
0924 01:01:45.443150 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:45.443208 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0924 01:01:45.443382 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:01:45.443735 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:01:45.443744 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:01:45.444441 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0924 01:01:45.444671 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:45.444679 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:45.446149 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:45.446153 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:45.447734 (+ 1581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:45.447746 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:45.448730 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:45.448737 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:45.448947 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:45.449597 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 01:01:45.449629 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0924 01:01:45.452281 (+ 2652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 01:01:45.455927 (+ 3646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:01:45.455936 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:01:45.455940 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:01:45.456349 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:01:45.456806 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:01:45.456810 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:01:45.456814 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:01:45.457008 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:01:45.457611 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 01:01:45.457616 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 01:01:45.457765 (+ 149us) client_negotiation.cc:769] Sending connection context
0924 01:01:45.457898 (+ 133us) client_negotiation.cc:240] Negotiation successful
0924 01:01:45.458129 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":311,"threads_started":1}
I20260924 01:01:45.458673 31362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:45.442506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53812 (local address 127.8.64.190:45487)
0924 01:01:45.442758 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:45.442767 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:45.443316 (+ 549us) server_negotiation.cc:407] Connection header received
0924 01:01:45.443492 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:45.443512 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:45.443575 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:45.443654 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:45.444818 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:45.445959 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:45.447878 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:45.448544 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:45.448635 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:45.452447 (+ 3812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:45.452451 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:45.452455 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:01:45.452486 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:45.455792 (+ 3306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:45.456478 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:45.456483 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:45.456487 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:45.456646 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:45.457115 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:45.457120 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:45.457123 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:45.457476 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:45.457635 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:45.458051 (+ 416us) server_negotiation.cc:299] Negotiation successful
0924 01:01:45.458328 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260924 01:01:45.459897 31507 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.64.189" port: 45161 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 01:01:45 dist-test-slave-mmhs krb5kdc[31279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211705, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.188@KRBTEST.COM for kudu/127.8.64.189@KRBTEST.COM
Sep 24 01:01:45 dist-test-slave-mmhs krb5kdc[31279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211703, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.190@KRBTEST.COM
I20260924 01:01:45.472925 31511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:45.460490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.189:45161 (local address 127.0.0.1:46234)
0924 01:01:45.460721 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:45.460738 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0924 01:01:45.460825 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:01:45.461125 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:01:45.461129 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:01:45.461481 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0924 01:01:45.461613 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:45.461620 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:45.462621 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:45.462629 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:45.463488 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:45.463500 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:45.464177 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:45.464182 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:45.464351 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:45.464900 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 01:01:45.464927 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0924 01:01:45.467126 (+ 2199us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 01:01:45.470441 (+ 3315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:01:45.470449 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:01:45.470456 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:01:45.470908 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:01:45.471282 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:01:45.471288 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:01:45.471294 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:01:45.471442 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:01:45.472005 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 01:01:45.472010 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 01:01:45.472160 (+ 150us) client_negotiation.cc:769] Sending connection context
0924 01:01:45.472292 (+ 132us) client_negotiation.cc:240] Negotiation successful
0924 01:01:45.472556 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140}
I20260924 01:01:45.472954 31437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:45.460666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46234 (local address 127.8.64.189:45161)
0924 01:01:45.460838 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:45.460844 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:45.460874 (+ 30us) server_negotiation.cc:407] Connection header received
0924 01:01:45.460935 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:45.460946 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:45.461007 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:45.461106 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:45.461752 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:45.462490 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:45.463631 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:45.464029 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:45.464150 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:45.467321 (+ 3171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:45.467328 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:45.467333 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:01:45.467382 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:45.470285 (+ 2903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:45.471025 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:45.471032 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:45.471035 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:45.471176 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:45.471554 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:45.471559 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:45.471563 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:45.471898 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:45.472044 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:45.472389 (+ 345us) server_negotiation.cc:299] Negotiation successful
0924 01:01:45.472586 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260924 01:01:45.473801 31507 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.64.188" port: 33393 } has no permanent_uuid. Determining permanent_uuid...
I20260924 01:01:45.474575 31362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:45.460260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53820 (local address 127.8.64.190:45487)
0924 01:01:45.460435 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:45.460441 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:45.460458 (+ 17us) server_negotiation.cc:407] Connection header received
0924 01:01:45.460505 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:45.460509 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:45.460574 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:45.460679 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:45.461783 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:45.462581 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:45.463764 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:45.464182 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:45.464309 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:45.468787 (+ 4478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:45.468791 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:45.468795 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:01:45.468824 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:45.471799 (+ 2975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:45.472421 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:45.472427 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:45.472431 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:45.472566 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:45.472917 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:45.472923 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:45.472926 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:45.473307 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:45.473546 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:45.474001 (+ 455us) server_negotiation.cc:299] Negotiation successful
0924 01:01:45.474210 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
Sep 24 01:01:45 dist-test-slave-mmhs krb5kdc[31279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211705, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.188@KRBTEST.COM for kudu/127.8.64.188@KRBTEST.COM
Sep 24 01:01:45 dist-test-slave-mmhs krb5kdc[31279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211703, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.64.189@KRBTEST.COM
I20260924 01:01:45.487881 31510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:45.474786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55436 (local address 127.8.64.188:33393)
0924 01:01:45.475135 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:45.475141 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:45.475160 (+ 19us) server_negotiation.cc:407] Connection header received
0924 01:01:45.475208 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:45.475216 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:45.475289 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:45.475392 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:45.476282 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:45.477004 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:45.478661 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:45.479273 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:45.479380 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:45.482744 (+ 3364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:45.482749 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:45.482752 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:01:45.482779 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:45.485252 (+ 2473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:45.486025 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:45.486032 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:45.486036 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:45.486189 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:45.486648 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:45.486651 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:45.486653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:45.486910 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:45.487076 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:45.487474 (+ 398us) server_negotiation.cc:299] Negotiation successful
0924 01:01:45.487635 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236}
I20260924 01:01:45.488220 31511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:45.474577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:33393 (local address 127.0.0.1:55436)
0924 01:01:45.474729 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:45.474747 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0924 01:01:45.474853 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:01:45.475457 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:01:45.475465 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:01:45.475809 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0924 01:01:45.475936 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:45.475963 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:45.477118 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:45.477125 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:45.478538 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:45.478550 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:45.479447 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:45.479453 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:45.479661 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:45.480170 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 01:01:45.480200 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0924 01:01:45.482575 (+ 2375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 01:01:45.485368 (+ 2793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:01:45.485373 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:01:45.485377 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:01:45.485896 (+ 519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:01:45.486341 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:01:45.486346 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:01:45.486350 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:01:45.486543 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:01:45.487046 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 01:01:45.487049 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 01:01:45.487222 (+ 173us) client_negotiation.cc:769] Sending connection context
0924 01:01:45.487385 (+ 163us) client_negotiation.cc:240] Negotiation successful
0924 01:01:45.487834 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":103}
I20260924 01:01:45.491317 31437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:45.476165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46236 (local address 127.8.64.189:45161)
0924 01:01:45.476330 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:45.476336 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:45.476361 (+ 25us) server_negotiation.cc:407] Connection header received
0924 01:01:45.476464 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:45.476469 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:45.476533 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:45.476626 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:45.478164 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:45.479154 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:45.480788 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:45.481172 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:45.481259 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:45.485325 (+ 4066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:45.485333 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:45.485338 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:01:45.485376 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:45.488670 (+ 3294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:45.489308 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:45.489315 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:45.489318 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:45.489452 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:45.489901 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:45.489907 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:45.489910 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:45.490172 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:45.490429 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:45.490723 (+ 294us) server_negotiation.cc:299] Negotiation successful
0924 01:01:45.490953 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260924 01:01:45.493821 31507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db82d452228b45cca1b2b4ce1a8a2ae0: Bootstrap starting.
I20260924 01:01:45.495858 8450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 01:01:45.497448 31507 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db82d452228b45cca1b2b4ce1a8a2ae0: Neither blocks nor log segments found. Creating new log.
I20260924 01:01:45.498750 31507 log.cc:824] T 00000000000000000000000000000000 P db82d452228b45cca1b2b4ce1a8a2ae0: Log is configured to *not* fsync() on all Append() calls
I20260924 01:01:45.498098 31334 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "a7eb9683ee6345649db8d50f4288fe71"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53820
I20260924 01:01:45.501443 31507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db82d452228b45cca1b2b4ce1a8a2ae0: No bootstrap required, opened a new log
I20260924 01:01:45.509059 31507 raft_consensus.cc:348] T 00000000000000000000000000000000 P db82d452228b45cca1b2b4ce1a8a2ae0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7eb9683ee6345649db8d50f4288fe71" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45487 } } peers { permanent_uuid: "c3f2125d04bf42eeb6c7099439dcdf1c" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 45161 } } peers { permanent_uuid: "db82d452228b45cca1b2b4ce1a8a2ae0" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 33393 } }
I20260924 01:01:45.509387 31507 raft_consensus.cc:729] T 00000000000000000000000000000000 P db82d452228b45cca1b2b4ce1a8a2ae0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db82d452228b45cca1b2b4ce1a8a2ae0, State: Initialized, Role: FOLLOWER
I20260924 01:01:45.509809 31507 consensus_queue.cc:260] T 00000000000000000000000000000000 P db82d452228b45cca1b2b4ce1a8a2ae0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7eb9683ee6345649db8d50f4288fe71" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45487 } } peers { permanent_uuid: "c3f2125d04bf42eeb6c7099439dcdf1c" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 45161 } } peers { permanent_uuid: "db82d452228b45cca1b2b4ce1a8a2ae0" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 33393 } }
I20260924 01:01:45.510740 31513 sys_catalog.cc:455] T 00000000000000000000000000000000 P db82d452228b45cca1b2b4ce1a8a2ae0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7eb9683ee6345649db8d50f4288fe71" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45487 } } peers { permanent_uuid: "c3f2125d04bf42eeb6c7099439dcdf1c" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 45161 } } peers { permanent_uuid: "db82d452228b45cca1b2b4ce1a8a2ae0" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 33393 } } }
I20260924 01:01:45.511124 31513 sys_catalog.cc:458] T 00000000000000000000000000000000 P db82d452228b45cca1b2b4ce1a8a2ae0 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 01:01:45.511662 31507 sys_catalog.cc:564] T 00000000000000000000000000000000 P db82d452228b45cca1b2b4ce1a8a2ae0 [sys.catalog]: configured and running, proceeding with master startup.
W20260924 01:01:45.520694 31524 catalog_manager.cc:1558] T 00000000000000000000000000000000 P db82d452228b45cca1b2b4ce1a8a2ae0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 01:01:45.520848 31524 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 01:01:45.518863 31440 master_runner.cc:186] Error getting master registration for 127.8.64.190:45487: OK, instance_id { permanent_uuid: "a7eb9683ee6345649db8d50f4288fe71" instance_seqno: 1790211704399770 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 01:01:45.529908 31440 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 01:01:45.541339 31359 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.64.188" port: 33393 } attempt: 4
Sep 24 01:01:45 dist-test-slave-mmhs krb5kdc[31279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211704, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.190@KRBTEST.COM for kudu/127.8.64.188@KRBTEST.COM
I20260924 01:01:45.556900 31510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:45.542263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55452 (local address 127.8.64.188:33393)
0924 01:01:45.542496 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:45.542502 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:45.542528 (+ 26us) server_negotiation.cc:407] Connection header received
0924 01:01:45.542601 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:45.542606 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:45.542687 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:45.542807 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:45.543732 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:45.544888 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:45.546701 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:45.547400 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:45.547533 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:45.551179 (+ 3646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:45.551186 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:45.551191 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:01:45.551243 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:45.554210 (+ 2967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:45.555052 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:45.555056 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:45.555059 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:45.555175 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:45.555590 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:45.555593 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:45.555595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:45.555927 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:45.556044 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:45.556486 (+ 442us) server_negotiation.cc:299] Negotiation successful
0924 01:01:45.556653 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260924 01:01:45.557112 31363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:45.542082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:33393 (local address 127.0.0.1:55452)
0924 01:01:45.542321 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:45.542343 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0924 01:01:45.542486 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:01:45.542861 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:01:45.542869 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:01:45.543346 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0924 01:01:45.543578 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:45.543589 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:45.545043 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:45.545048 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:45.546560 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:45.546576 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:45.547547 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:45.547552 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:45.547770 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:45.548461 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 01:01:45.548480 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0924 01:01:45.550927 (+ 2447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 01:01:45.554384 (+ 3457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:01:45.554389 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:01:45.554394 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:01:45.554922 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:01:45.555275 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:01:45.555280 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:01:45.555284 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:01:45.555481 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:01:45.556042 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 01:01:45.556047 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 01:01:45.556218 (+ 171us) client_negotiation.cc:769] Sending connection context
0924 01:01:45.556369 (+ 151us) client_negotiation.cc:240] Negotiation successful
0924 01:01:45.556713 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260924 01:01:45.563473 31359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7eb9683ee6345649db8d50f4288fe71: Bootstrap starting.
I20260924 01:01:45.568125 31359 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7eb9683ee6345649db8d50f4288fe71: Neither blocks nor log segments found. Creating new log.
I20260924 01:01:45.569579 31359 log.cc:824] T 00000000000000000000000000000000 P a7eb9683ee6345649db8d50f4288fe71: Log is configured to *not* fsync() on all Append() calls
I20260924 01:01:45.573408 31359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7eb9683ee6345649db8d50f4288fe71: No bootstrap required, opened a new log
I20260924 01:01:45.581557 31359 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7eb9683ee6345649db8d50f4288fe71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7eb9683ee6345649db8d50f4288fe71" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45487 } } peers { permanent_uuid: "c3f2125d04bf42eeb6c7099439dcdf1c" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 45161 } } peers { permanent_uuid: "db82d452228b45cca1b2b4ce1a8a2ae0" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 33393 } }
I20260924 01:01:45.582053 31359 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7eb9683ee6345649db8d50f4288fe71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7eb9683ee6345649db8d50f4288fe71, State: Initialized, Role: FOLLOWER
I20260924 01:01:45.582630 31359 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7eb9683ee6345649db8d50f4288fe71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7eb9683ee6345649db8d50f4288fe71" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45487 } } peers { permanent_uuid: "c3f2125d04bf42eeb6c7099439dcdf1c" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 45161 } } peers { permanent_uuid: "db82d452228b45cca1b2b4ce1a8a2ae0" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 33393 } }
I20260924 01:01:45.583717 31527 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7eb9683ee6345649db8d50f4288fe71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7eb9683ee6345649db8d50f4288fe71" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45487 } } peers { permanent_uuid: "c3f2125d04bf42eeb6c7099439dcdf1c" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 45161 } } peers { permanent_uuid: "db82d452228b45cca1b2b4ce1a8a2ae0" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 33393 } } }
I20260924 01:01:45.584100 31527 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7eb9683ee6345649db8d50f4288fe71 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 01:01:45.585073 31359 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7eb9683ee6345649db8d50f4288fe71 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 01:01:45.585109 31334 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7eb9683ee6345649db8d50f4288fe71 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 01:01:45.585322 31334 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7eb9683ee6345649db8d50f4288fe71 [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:01:45.588236 31334 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7eb9683ee6345649db8d50f4288fe71 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7eb9683ee6345649db8d50f4288fe71" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45487 } } peers { permanent_uuid: "c3f2125d04bf42eeb6c7099439dcdf1c" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 45161 } } peers { permanent_uuid: "db82d452228b45cca1b2b4ce1a8a2ae0" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 33393 } }
I20260924 01:01:45.590485 31482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a7eb9683ee6345649db8d50f4288fe71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "db82d452228b45cca1b2b4ce1a8a2ae0"
I20260924 01:01:45.591006 31482 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db82d452228b45cca1b2b4ce1a8a2ae0 [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:01:45.594441 31482 raft_consensus.cc:2454] T 00000000000000000000000000000000 P db82d452228b45cca1b2b4ce1a8a2ae0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7eb9683ee6345649db8d50f4288fe71 in term 1.
I20260924 01:01:45.594833 31334 leader_election.cc:288] T 00000000000000000000000000000000 P a7eb9683ee6345649db8d50f4288fe71 [CANDIDATE]: Term 1 election: Requested vote from peers c3f2125d04bf42eeb6c7099439dcdf1c (127.8.64.189:45161), db82d452228b45cca1b2b4ce1a8a2ae0 (127.8.64.188:33393)
I20260924 01:01:45.595151 31309 leader_election.cc:302] T 00000000000000000000000000000000 P a7eb9683ee6345649db8d50f4288fe71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7eb9683ee6345649db8d50f4288fe71, db82d452228b45cca1b2b4ce1a8a2ae0; no voters:
I20260924 01:01:45.595664 31527 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7eb9683ee6345649db8d50f4288fe71 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 01:01:45.596282 31527 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7eb9683ee6345649db8d50f4288fe71 [term 1 LEADER]: Becoming Leader. State: Replica: a7eb9683ee6345649db8d50f4288fe71, State: Running, Role: LEADER
I20260924 01:01:45.596809 31527 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7eb9683ee6345649db8d50f4288fe71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7eb9683ee6345649db8d50f4288fe71" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45487 } } peers { permanent_uuid: "c3f2125d04bf42eeb6c7099439dcdf1c" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 45161 } } peers { permanent_uuid: "db82d452228b45cca1b2b4ce1a8a2ae0" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 33393 } }
I20260924 01:01:45.600600 31532 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7eb9683ee6345649db8d50f4288fe71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7eb9683ee6345649db8d50f4288fe71. Latest consensus state: current_term: 1 leader_uuid: "a7eb9683ee6345649db8d50f4288fe71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7eb9683ee6345649db8d50f4288fe71" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45487 } } peers { permanent_uuid: "c3f2125d04bf42eeb6c7099439dcdf1c" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 45161 } } peers { permanent_uuid: "db82d452228b45cca1b2b4ce1a8a2ae0" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 33393 } } }
I20260924 01:01:45.601173 31532 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7eb9683ee6345649db8d50f4288fe71 [sys.catalog]: This master's current role is: LEADER
I20260924 01:01:45.606985 31535 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 01:01:45.613168 31535 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 01:01:45.631419 31482 raft_consensus.cc:1261] T 00000000000000000000000000000000 P db82d452228b45cca1b2b4ce1a8a2ae0 [term 1 FOLLOWER]: Refusing update from remote peer a7eb9683ee6345649db8d50f4288fe71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 01:01:45.632458 31532 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a7eb9683ee6345649db8d50f4288fe71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db82d452228b45cca1b2b4ce1a8a2ae0" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 33393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 01:01:45.633991 31433 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.64.188" port: 33393 } attempt: 5
W20260924 01:01:45.639237 31313 tablet.cc:2369] T 00000000000000000000000000000000 P a7eb9683ee6345649db8d50f4288fe71: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 24 01:01:45 dist-test-slave-mmhs krb5kdc[31279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790211704, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.64.189@KRBTEST.COM for kudu/127.8.64.188@KRBTEST.COM
I20260924 01:01:45.653568 31436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:45.634580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.64.188:33393 (local address 127.0.0.1:55456)
0924 01:01:45.634926 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0924 01:01:45.634947 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0924 01:01:45.635059 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 01:01:45.635853 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 01:01:45.635858 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 01:01:45.636410 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0924 01:01:45.636606 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:45.636617 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:45.638710 (+ 2093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:45.638718 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:45.640468 (+ 1750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 01:01:45.640486 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 01:01:45.641806 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 01:01:45.641814 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 01:01:45.642114 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:45.642860 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 01:01:45.642883 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0924 01:01:45.646118 (+ 3235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 01:01:45.650497 (+ 4379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:01:45.650506 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:01:45.650511 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:01:45.651068 (+ 557us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:01:45.651552 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 01:01:45.651559 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 01:01:45.651566 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0924 01:01:45.651742 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 01:01:45.652485 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 01:01:45.652493 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 01:01:45.652664 (+ 171us) client_negotiation.cc:769] Sending connection context
0924 01:01:45.652820 (+ 156us) client_negotiation.cc:240] Negotiation successful
0924 01:01:45.653116 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216}
I20260924 01:01:45.654623 31510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:45.635242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55456 (local address 127.8.64.188:33393)
0924 01:01:45.635408 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:45.635415 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:45.635434 (+ 19us) server_negotiation.cc:407] Connection header received
0924 01:01:45.635484 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:45.635489 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:45.635560 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:45.635663 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:45.636808 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:45.638123 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:45.640668 (+ 2545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:45.641445 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:45.641578 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:45.646335 (+ 4757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:45.646354 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:45.646363 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:01:45.646411 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:45.650286 (+ 3875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:45.651211 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:45.651217 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:45.651221 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:45.651412 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:45.651875 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:45.651883 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:45.651888 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:45.652256 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:45.652417 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:45.653956 (+ 1539us) server_negotiation.cc:299] Negotiation successful
0924 01:01:45.654209 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260924 01:01:45.646859 31292 master_runner.cc:186] Error getting master registration for 127.8.64.189:45161: OK, instance_id { permanent_uuid: "c3f2125d04bf42eeb6c7099439dcdf1c" instance_seqno: 1790211704910546 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 01:01:45.658700 31292 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260924 01:01:45.660853 31461 tablet.cc:2369] T 00000000000000000000000000000000 P db82d452228b45cca1b2b4ce1a8a2ae0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 01:01:45.661137 31433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3f2125d04bf42eeb6c7099439dcdf1c: Bootstrap starting.
I20260924 01:01:45.665944 31433 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3f2125d04bf42eeb6c7099439dcdf1c: Neither blocks nor log segments found. Creating new log.
I20260924 01:01:45.674496 31433 log.cc:824] T 00000000000000000000000000000000 P c3f2125d04bf42eeb6c7099439dcdf1c: Log is configured to *not* fsync() on all Append() calls
I20260924 01:01:45.678839 31513 sys_catalog.cc:455] T 00000000000000000000000000000000 P db82d452228b45cca1b2b4ce1a8a2ae0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7eb9683ee6345649db8d50f4288fe71. Latest consensus state: current_term: 1 leader_uuid: "a7eb9683ee6345649db8d50f4288fe71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7eb9683ee6345649db8d50f4288fe71" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45487 } } peers { permanent_uuid: "c3f2125d04bf42eeb6c7099439dcdf1c" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 45161 } } peers { permanent_uuid: "db82d452228b45cca1b2b4ce1a8a2ae0" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 33393 } } }
I20260924 01:01:45.679198 31513 sys_catalog.cc:458] T 00000000000000000000000000000000 P db82d452228b45cca1b2b4ce1a8a2ae0 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 01:01:45.681560 31545 mvcc.cc:205] Tried to move back new op lower bound from 7332707146241736704 to 7332707146128797696. Current Snapshot: MvccSnapshot[applied={T|T < 7332707146241736704}]
I20260924 01:01:45.686798 31532 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7eb9683ee6345649db8d50f4288fe71 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a7eb9683ee6345649db8d50f4288fe71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7eb9683ee6345649db8d50f4288fe71" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45487 } } peers { permanent_uuid: "c3f2125d04bf42eeb6c7099439dcdf1c" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 45161 } } peers { permanent_uuid: "db82d452228b45cca1b2b4ce1a8a2ae0" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 33393 } } }
I20260924 01:01:45.687199 31532 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7eb9683ee6345649db8d50f4288fe71 [sys.catalog]: This master's current role is: LEADER
I20260924 01:01:45.688515 31548 mvcc.cc:205] Tried to move back new op lower bound from 7332707146241736704 to 7332707146128797696. Current Snapshot: MvccSnapshot[applied={T|T < 7332707146241736704}]
I20260924 01:01:45.688657 31535 catalog_manager.cc:1347] Generated new cluster ID: 6f0000e5ab544f8a8ce513186589a1d8
I20260924 01:01:45.688757 31535 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 01:01:45.698230 31513 sys_catalog.cc:455] T 00000000000000000000000000000000 P db82d452228b45cca1b2b4ce1a8a2ae0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a7eb9683ee6345649db8d50f4288fe71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7eb9683ee6345649db8d50f4288fe71" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45487 } } peers { permanent_uuid: "c3f2125d04bf42eeb6c7099439dcdf1c" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 45161 } } peers { permanent_uuid: "db82d452228b45cca1b2b4ce1a8a2ae0" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 33393 } } }
I20260924 01:01:45.698566 31513 sys_catalog.cc:458] T 00000000000000000000000000000000 P db82d452228b45cca1b2b4ce1a8a2ae0 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 01:01:45.705329 31433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3f2125d04bf42eeb6c7099439dcdf1c: No bootstrap required, opened a new log
I20260924 01:01:45.723800 31510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:45.666085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55470 (local address 127.8.64.188:33393)
0924 01:01:45.666471 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:45.666480 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:45.690868 (+ 24388us) server_negotiation.cc:407] Connection header received
0924 01:01:45.692462 (+ 1594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:45.692469 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:45.692568 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:45.692645 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:45.698928 (+ 6283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:45.699905 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:45.711835 (+ 11930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:45.712565 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:45.712677 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:45.716128 (+ 3451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:45.716135 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:45.716141 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:01:45.716184 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:45.720064 (+ 3880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:45.720877 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:45.720884 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:45.720887 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:45.721038 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:45.721488 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:45.721493 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:45.721496 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:45.721853 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:45.722105 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:45.723139 (+ 1034us) server_negotiation.cc:299] Negotiation successful
0924 01:01:45.723382 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260924 01:01:45.728294 31433 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3f2125d04bf42eeb6c7099439dcdf1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7eb9683ee6345649db8d50f4288fe71" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45487 } } peers { permanent_uuid: "c3f2125d04bf42eeb6c7099439dcdf1c" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 45161 } } peers { permanent_uuid: "db82d452228b45cca1b2b4ce1a8a2ae0" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 33393 } }
I20260924 01:01:45.728753 31433 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3f2125d04bf42eeb6c7099439dcdf1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3f2125d04bf42eeb6c7099439dcdf1c, State: Initialized, Role: FOLLOWER
I20260924 01:01:45.729418 31433 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3f2125d04bf42eeb6c7099439dcdf1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7eb9683ee6345649db8d50f4288fe71" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45487 } } peers { permanent_uuid: "c3f2125d04bf42eeb6c7099439dcdf1c" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 45161 } } peers { permanent_uuid: "db82d452228b45cca1b2b4ce1a8a2ae0" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 33393 } }
I20260924 01:01:45.732161 31362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:45.658103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53826 (local address 127.8.64.190:45487)
0924 01:01:45.658480 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:45.658487 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:45.683521 (+ 25034us) server_negotiation.cc:407] Connection header received
0924 01:01:45.684127 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:45.684141 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:45.684257 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:45.684354 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:45.688060 (+ 3706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:45.689453 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:45.695889 (+ 6436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:45.696633 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:45.696754 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:45.713357 (+ 16603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:45.713365 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:45.713370 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:01:45.713426 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:45.724099 (+ 10673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:45.724927 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:45.724934 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:45.724938 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:45.725128 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:45.725577 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:45.725582 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:45.725585 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:45.725983 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:45.726232 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:45.731484 (+ 5252us) server_negotiation.cc:299] Negotiation successful
0924 01:01:45.731734 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226}
I20260924 01:01:45.735216 31437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:45.691351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46244 (local address 127.8.64.189:45161)
0924 01:01:45.691626 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:45.691635 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:45.698875 (+ 7240us) server_negotiation.cc:407] Connection header received
0924 01:01:45.710588 (+ 11713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:45.710595 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:45.710670 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:45.710738 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:45.712735 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:45.713821 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:45.715826 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:45.716519 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:45.716613 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:45.720083 (+ 3470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 01:01:45.720090 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 01:01:45.720096 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 01:01:45.720145 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0924 01:01:45.729562 (+ 9417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:45.730384 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:45.730391 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:45.730397 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:45.730563 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 01:01:45.732777 (+ 2214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 01:01:45.732784 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 01:01:45.732787 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0924 01:01:45.733176 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 01:01:45.733425 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:45.734594 (+ 1169us) server_negotiation.cc:299] Negotiation successful
0924 01:01:45.734845 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260924 01:01:45.736509 31556 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3f2125d04bf42eeb6c7099439dcdf1c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7eb9683ee6345649db8d50f4288fe71" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45487 } } peers { permanent_uuid: "c3f2125d04bf42eeb6c7099439dcdf1c" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 45161 } } peers { permanent_uuid: "db82d452228b45cca1b2b4ce1a8a2ae0" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 33393 } } }
I20260924 01:01:45.738915 31556 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3f2125d04bf42eeb6c7099439dcdf1c [sys.catalog]: This master's current role is: FOLLOWER
I20260924 01:01:45.738781 31433 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3f2125d04bf42eeb6c7099439dcdf1c [sys.catalog]: configured and running, proceeding with master startup.
I20260924 01:01:45.738852 31408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a7eb9683ee6345649db8d50f4288fe71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c3f2125d04bf42eeb6c7099439dcdf1c"
I20260924 01:01:45.739261 31407 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3f2125d04bf42eeb6c7099439dcdf1c [term 0 FOLLOWER]: Advancing to term 1
I20260924 01:01:45.742843 31407 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c3f2125d04bf42eeb6c7099439dcdf1c [term 1 FOLLOWER]: Refusing update from remote peer a7eb9683ee6345649db8d50f4288fe71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 01:01:45.743733 31532 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a7eb9683ee6345649db8d50f4288fe71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3f2125d04bf42eeb6c7099439dcdf1c" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 45161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 01:01:45.746325 31408 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c3f2125d04bf42eeb6c7099439dcdf1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7eb9683ee6345649db8d50f4288fe71 in term 1.
I20260924 01:01:45.759523 31535 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 01:01:45.761381 31535 catalog_manager.cc:1504] Loading token signing keys...
I20260924 01:01:45.782039 31535 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7eb9683ee6345649db8d50f4288fe71: Generated new TSK 0
I20260924 01:01:45.783056 31535 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 01:01:45.784058 31532 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7eb9683ee6345649db8d50f4288fe71 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a7eb9683ee6345649db8d50f4288fe71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7eb9683ee6345649db8d50f4288fe71" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45487 } } peers { permanent_uuid: "c3f2125d04bf42eeb6c7099439dcdf1c" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 45161 } } peers { permanent_uuid: "db82d452228b45cca1b2b4ce1a8a2ae0" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 33393 } } }
I20260924 01:01:45.784403 31532 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7eb9683ee6345649db8d50f4288fe71 [sys.catalog]: This master's current role is: LEADER
I20260924 01:01:45.785560 31556 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3f2125d04bf42eeb6c7099439dcdf1c [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7eb9683ee6345649db8d50f4288fe71. Latest consensus state: current_term: 1 leader_uuid: "a7eb9683ee6345649db8d50f4288fe71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7eb9683ee6345649db8d50f4288fe71" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45487 } } peers { permanent_uuid: "c3f2125d04bf42eeb6c7099439dcdf1c" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 45161 } } peers { permanent_uuid: "db82d452228b45cca1b2b4ce1a8a2ae0" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 33393 } } }
I20260924 01:01:45.785539 31562 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3f2125d04bf42eeb6c7099439dcdf1c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a7eb9683ee6345649db8d50f4288fe71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7eb9683ee6345649db8d50f4288fe71" member_type: VOTER last_known_addr { host: "127.8.64.190" port: 45487 } } peers { permanent_uuid: "c3f2125d04bf42eeb6c7099439dcdf1c" member_type: VOTER last_known_addr { host: "127.8.64.189" port: 45161 } } peers { permanent_uuid: "db82d452228b45cca1b2b4ce1a8a2ae0" member_type: VOTER last_known_addr { host: "127.8.64.188" port: 33393 } } }
I20260924 01:01:45.785898 31556 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3f2125d04bf42eeb6c7099439dcdf1c [sys.catalog]: This master's current role is: FOLLOWER
I20260924 01:01:45.785939 31562 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3f2125d04bf42eeb6c7099439dcdf1c [sys.catalog]: This master's current role is: FOLLOWER
I20260924 01:01:45.787567 31558 mvcc.cc:205] Tried to move back new op lower bound from 7332707146241736704 to 7332707146128797696. Current Snapshot: MvccSnapshot[applied={T|T < 7332707146241736704}]
I20260924 01:01:45.849323 31573 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260924 01:01:45.884223 31510 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 01:01:45.868675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55478 (local address 127.8.64.188:33393)
0924 01:01:45.869015 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:45.869022 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:45.881721 (+ 12699us) server_negotiation.cc:407] Connection header received
0924 01:01:45.882614 (+ 893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:45.882622 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:45.882714 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:45.882791 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:45.883961 (+ 1170us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55478: BlockingRecv error: recv got EOF from 127.0.0.1:55478 (error 108)
Metrics: {"server-negotiator.queue_time_us":221}
W20260924 01:01:45.886215 31362 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 01:01:45.868503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53832 (local address 127.8.64.190:45487)
0924 01:01:45.868864 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:45.868875 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:45.881210 (+ 12335us) server_negotiation.cc:407] Connection header received
0924 01:01:45.884548 (+ 3338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:45.884554 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:45.884660 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:45.884730 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:45.885977 (+ 1247us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53832: BlockingRecv error: recv got EOF from 127.0.0.1:53832 (error 108)
Metrics: {"server-negotiator.queue_time_us":236}
W20260924 01:01:45.888664 31437 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 01:01:45.882075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46250 (local address 127.8.64.189:45161)
0924 01:01:45.882265 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:45.882271 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:45.887095 (+ 4824us) server_negotiation.cc:407] Connection header received
0924 01:01:45.887424 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:45.887430 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:45.887512 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:45.887583 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:45.888455 (+ 872us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46250: BlockingRecv error: recv got EOF from 127.0.0.1:46250 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260924 01:01:45.895462 31437 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 01:01:45.893775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46262 (local address 127.8.64.189:45161)
0924 01:01:45.893935 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:45.893965 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:45.893984 (+ 19us) server_negotiation.cc:407] Connection header received
0924 01:01:45.894257 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:45.894264 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:45.894411 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:45.894519 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:45.895209 (+ 690us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46262: BlockingRecv error: recv got EOF from 127.0.0.1:46262 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260924 01:01:45.901733 31437 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 01:01:45.899658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46266 (local address 127.8.64.189:45161)
0924 01:01:45.899811 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:45.899817 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:45.900254 (+ 437us) server_negotiation.cc:407] Connection header received
0924 01:01:45.900508 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:45.900515 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:45.900599 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:45.900671 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:45.901531 (+ 860us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46266: BlockingRecv error: recv got EOF from 127.0.0.1:46266 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260924 01:01:45.913381 31437 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 01:01:45.908009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46282 (local address 127.8.64.189:45161)
0924 01:01:45.908164 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:45.908170 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:45.911932 (+ 3762us) server_negotiation.cc:407] Connection header received
0924 01:01:45.912137 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:45.912142 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:45.912268 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:45.912393 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:45.913145 (+ 752us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46282: BlockingRecv error: recv got EOF from 127.0.0.1:46282 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260924 01:01:45.960255 31437 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 01:01:45.954890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46298 (local address 127.8.64.189:45161)
0924 01:01:45.955126 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:45.955132 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:45.958895 (+ 3763us) server_negotiation.cc:407] Connection header received
0924 01:01:45.959115 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:45.959120 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:45.959200 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:45.959310 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:45.960083 (+ 773us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46298: BlockingRecv error: recv got EOF from 127.0.0.1:46298 (error 108)
Metrics: {"server-negotiator.queue_time_us":119}
W20260924 01:01:45.974869 31437 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 01:01:45.972705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46314 (local address 127.8.64.189:45161)
0924 01:01:45.972914 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:45.972921 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:45.973431 (+ 510us) server_negotiation.cc:407] Connection header received
0924 01:01:45.973644 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:45.973654 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:45.973726 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:45.973842 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:45.974629 (+ 787us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46314: BlockingRecv error: recv got EOF from 127.0.0.1:46314 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260924 01:01:45.996621 31437 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 01:01:45.994412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46328 (local address 127.8.64.189:45161)
0924 01:01:45.994634 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:45.994644 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:45.995181 (+ 537us) server_negotiation.cc:407] Connection header received
0924 01:01:45.995408 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:45.995412 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:45.995487 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:45.995597 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:45.996376 (+ 779us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46328: BlockingRecv error: recv got EOF from 127.0.0.1:46328 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260924 01:01:46.034194 31437 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 01:01:46.031922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46336 (local address 127.8.64.189:45161)
0924 01:01:46.032150 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:46.032160 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:46.032731 (+ 571us) server_negotiation.cc:407] Connection header received
0924 01:01:46.032954 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:46.032960 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:46.033040 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:46.033148 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:46.033923 (+ 775us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46336: BlockingRecv error: recv got EOF from 127.0.0.1:46336 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260924 01:01:46.105548 31437 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 01:01:46.103246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46352 (local address 127.8.64.189:45161)
0924 01:01:46.103523 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:46.103532 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:46.104096 (+ 564us) server_negotiation.cc:407] Connection header received
0924 01:01:46.104327 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:46.104331 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:46.104463 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:46.104601 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:46.105311 (+ 710us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46352: BlockingRecv error: recv got EOF from 127.0.0.1:46352 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
W20260924 01:01:46.238827 31437 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 01:01:46.236881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46354 (local address 127.8.64.189:45161)
0924 01:01:46.237122 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:46.237129 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:46.237351 (+ 222us) server_negotiation.cc:407] Connection header received
0924 01:01:46.237600 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:46.237606 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:46.237685 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:46.237793 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:46.238568 (+ 775us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46354: BlockingRecv error: recv got EOF from 127.0.0.1:46354 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260924 01:01:46.502139 31437 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 01:01:46.500141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46370 (local address 127.8.64.189:45161)
0924 01:01:46.500373 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:46.500381 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:46.500588 (+ 207us) server_negotiation.cc:407] Connection header received
0924 01:01:46.500854 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:46.500859 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:46.500938 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:46.501059 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0924 01:01:46.501897 (+ 838us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46370: BlockingRecv error: recv got EOF from 127.0.0.1:46370 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260924 01:01:46.522857 31524 catalog_manager.cc:1259] Loaded cluster ID: 6f0000e5ab544f8a8ce513186589a1d8
I20260924 01:01:46.522970 31524 catalog_manager.cc:1552] T 00000000000000000000000000000000 P db82d452228b45cca1b2b4ce1a8a2ae0: loading cluster ID for follower catalog manager: success
I20260924 01:01:46.525400 31524 catalog_manager.cc:1574] T 00000000000000000000000000000000 P db82d452228b45cca1b2b4ce1a8a2ae0: acquiring CA information for follower catalog manager: success
I20260924 01:01:46.527211 31524 catalog_manager.cc:1602] T 00000000000000000000000000000000 P db82d452228b45cca1b2b4ce1a8a2ae0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 01:01:46.853727 31573 catalog_manager.cc:1259] Loaded cluster ID: 6f0000e5ab544f8a8ce513186589a1d8
I20260924 01:01:46.853865 31573 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c3f2125d04bf42eeb6c7099439dcdf1c: loading cluster ID for follower catalog manager: success
I20260924 01:01:46.856546 31573 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c3f2125d04bf42eeb6c7099439dcdf1c: acquiring CA information for follower catalog manager: success
I20260924 01:01:46.858394 31573 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c3f2125d04bf42eeb6c7099439dcdf1c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 01:01:47.024369 31638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:47.017867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46384 (local address 127.8.64.189:45161)
0924 01:01:47.018744 (+ 877us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:47.018748 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:47.018764 (+ 16us) server_negotiation.cc:407] Connection header received
0924 01:01:47.018834 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:47.018838 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:47.019007 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:47.019157 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 01:01:47.019606 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:47.020450 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:47.021876 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:47.022420 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:47.022556 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:47.023202 (+ 646us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 01:01:47.023739 (+ 537us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 01:01:47.023918 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:47.024000 (+ 82us) server_negotiation.cc:299] Negotiation successful
0924 01:01:47.024084 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":698,"thread_start_us":524,"threads_started":1}
I20260924 01:01:47.043939 31644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:47.032192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55494 (local address 127.8.64.188:33393)
0924 01:01:47.038684 (+ 6492us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:47.038691 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:47.038715 (+ 24us) server_negotiation.cc:407] Connection header received
0924 01:01:47.038819 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:47.038824 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:47.038990 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:47.039193 (+ 203us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 01:01:47.039666 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:47.040393 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:47.041742 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:47.042134 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:47.042265 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:47.042791 (+ 526us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 01:01:47.043335 (+ 544us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 01:01:47.043492 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:47.043550 (+ 58us) server_negotiation.cc:299] Negotiation successful
0924 01:01:47.043632 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6382,"thread_start_us":6289,"threads_started":1}
I20260924 01:01:47.071310 31650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:47.050064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53834 (local address 127.8.64.190:45487)
0924 01:01:47.062428 (+ 12364us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:47.062438 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:47.063207 (+ 769us) server_negotiation.cc:407] Connection header received
0924 01:01:47.063341 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:47.063346 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:47.063558 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:47.063761 (+ 203us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 01:01:47.064391 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:47.065547 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:47.067932 (+ 2385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:47.068567 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:47.068670 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:47.069499 (+ 829us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 01:01:47.070081 (+ 582us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 01:01:47.070175 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:47.070785 (+ 610us) server_negotiation.cc:299] Negotiation successful
0924 01:01:47.070891 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12164,"thread_start_us":12384,"threads_started":1}
I20260924 01:01:47.072237 31644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:47.050657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55508 (local address 127.8.64.188:33393)
0924 01:01:47.050870 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:47.050879 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:47.063147 (+ 12268us) server_negotiation.cc:407] Connection header received
0924 01:01:47.064312 (+ 1165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:47.064322 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:47.064452 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:47.064523 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 01:01:47.065493 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:47.066967 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:47.069949 (+ 2982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:47.070622 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:47.070738 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:47.071442 (+ 704us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 01:01:47.071570 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 01:01:47.071630 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:47.071879 (+ 249us) server_negotiation.cc:299] Negotiation successful
0924 01:01:47.071971 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260924 01:01:47.076468 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 31292
I20260924 01:01:47.077569 31354 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 01:01:47.077982 31638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 01:01:47.063344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46388 (local address 127.8.64.189:45161)
0924 01:01:47.063551 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0924 01:01:47.063558 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 01:01:47.070681 (+ 7123us) server_negotiation.cc:407] Connection header received
0924 01:01:47.071172 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 01:01:47.071179 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 01:01:47.071271 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 01:01:47.071338 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 01:01:47.072788 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:47.073957 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:47.075788 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 01:01:47.076466 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 01:01:47.076609 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 01:01:47.077176 (+ 567us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 01:01:47.077311 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 01:01:47.077420 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0924 01:01:47.077589 (+ 169us) server_negotiation.cc:299] Negotiation successful
0924 01:01:47.077673 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260924 01:01:47.190023 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 31292
I20260924 01:01:47.217532 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 31366
I20260924 01:01:47.218696 31428 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 01:01:47.331581 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 31366
I20260924 01:01:47.352299 8450 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu pid 31440
I20260924 01:01:47.353530 31502 generic_service.cc:196] Checking for leaks (request via RPC)
I20260924 01:01:47.459579 8450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDpZYBj/build/asan/bin/kudu with pid 31440
2026-09-24T01:01:47Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4878 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9496 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (364327 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST